For over 25 years Human Inference has collected reference data, for instance on persons. Because of our reference set we immediately recognize that ‘Ask Me’ and ‘Any Body’ are fake names. People are using these either in test situations or to hide their actual names.

In the old days we only needed to test on ‘Test Test’, in more recent years we see great inventiveness on these fake names. A brief example can be seen in the following list.

In case you cannot rely on reference data and interpretation you need to provide a check list. Providing it is one thing, but since users tend to be really creative, maintaining it is essential.

In these 25 years we identified a move from ‘real fake names’ towards ‘real names used in a fake way’. In the USA, for example, we identified popular Hollywood names and names of politicians being used as fake names. Currently the usage of the name ‘George Bush’ is decreasing, whereas ‘Barack Obama’ is increasingly used. We recognize the false usage of these names because of the change in frequency figures of the given name and family name as well as the usage of the combination itself. Remarkable is that ‘Abraham Lincoln’ and ‘George Washington’ are quite steady.

Back to ‘Walter von Stolzing’. By now you might have guessed what is happening here. We recognized that in German speaking areas this name is also passing our threshold on validity. By googling the name you can see that Walter is actually a character in Wagner’s opera ‘Die Meistersinger von Nürnberg’ back from 1868!

Let’s see if in 100 years time people are still using ‘Darth Vader’, ‘Lord Rings’ or ‘Snoop Dogg’!

All the names used in this blog are ‘real’ names coming from a popular social media site. Please check our data cleansing products in case you need cleansing solutions.

Your parents probably gave a lot of thought to the name they once gave you, and as it turns out they were right to do so! Research tells us a name can do wonders for its owner, as well as a lot of damage for that matter. Let’s have a look at some remarkable results.

Peter for President!
Recent studies show that in the US a student called Fred is more likely to fail his exam than a student who just happened to be named Andrew: people tend to indentify with their name and, in general, have a positive feeling about letters that correspond with their initials. Consequently Fred is far more likely to settle for a meager F, while Andrew will have an extra motive to strive for an A. It also explains how in choosing a partner we show a slight preference for someone whose name resembles our own, or why Mary will prefer to live in Maryland, while Monica is more inclined to settle in Santa Monica. Most of these preferences only show themselves through our subliminal selves, so we are not actually aware of the motivation for some of our choises. Another US study endorses these findings: inspired by the results mentioned above, researchers decided they’d investigate on another letter. They came up with the letter K, which in baseball stands for strikeout. The study showed once again that there is a connection between a letter and its causer: batters whose names began with a K struck out more often than other batters.

Ominous names
A UK research tells us that as much as one in 5 parents regret how they named their child. The novelty might have worn off after a few years, but can there be any real objections to a certain name? Apparently, there are plenty! Ironically it’s not the parents who’ll have to carry this burden for the rest of their lives…

“Hi, I’m Antwan, but you can call me Antoine…”
It seems that even children’s language skills are influenced by their name. This has to do with the effect negative emotions can have on a child’s performance. If for example you decided to name your son ‘Gene’ but spell it ‘Jene’, he is very likely to get confronted with disbelief from his teachers. “Are you sure your name isn’t spelled with a ‘G’?” This can severely undermine Jene’s sense of confidence. That explains why children with an unusual name or a name that is unusually spelled generally are less adequate spellers and readers.

“But Sissi is a Royal name, dear!”
When a girl is called Frankie we think it’s a fun name, a cool and robust statement to fit a strong personality. Yet when a boy is called Mckenzie, (yes, some parents think it’s cute to give their boy a name that has a feminine touch to it ) we see a similar effect, but with a different outcome. This is something his parents obviously had not foreseen: their son will constantly be shaking off his girly image. The effect is striking: boys with a androgynous name misbehave more often than their unambiguously named peers, especially when they reach puberty. A boy called Mckenzie or Aubrey is even more likely to display bad behaviour when there is a girl with the same name among his peers. One more reason for parents to stick to conventions when choosing a name for their newborn.

Want to produce the new Einstein? Call her Kate!
A name can be a burden, but if you use this knowledge wisely, you might just turn it into an advantage. What happens to a girl when she has finished school and needs to choose what subject to study? Well, according to a US study, her choice depends on her name. As it turns out girls with a very feminine name like Julietta or Isabella are more likely to study humanities, while those whose name is less obviously feminine are more partial towards science. The question is: who’s aspiring to whom? Could it be that parents would treat Kate in a different way than Barbara? Or did the parents subconciously decide they wanted to raise a scientist when they decided to call their daughter Kate?

Would you rather hire Vanity or Grace?
Of course it’s not just letters or gender that determines how we feel about a name. In fact, how other people perceive us very much depends on the meaning of our name. For example: when looking for a new member on your marketing team, would you rather hire Vanity or Grace? In spite of what her name tells us, Grace might be a job jumper who doesn’t know how to work in unison with her colleagues. Vanity on the other hand could just be a daughter of a well-read mother who had just finished her latest Thackeray when she gave birth. Still, both women will either meet a lot of prejudice or feel the need to live up to a very high standard because of their name.

It all goes to show that a name defenitely posesses some self-fulfilling qualities. Given the fact that so many parents regret their choice of names afterwards makes me think that the owners of that name might share these sentiments. So what does that mean when looking at it from a data quality point of view? Unisex names for example are responsible for a lot of data quality issues. As the borders between male and female names are fading we’ll need to update our knowledge continually. The human in Human Inference will definitely take care of that. After all, we wouldn’t want to you to put off Mrs Clinton when sending her a petition to take pity on the Syrian citizens starting: “Dear Mr. Clinton…”.

Source: Livescience.com & Babynames.com

The internet is an ocean of wealthy content, but unfortunately, as in the real world, it’s heavily polluted.

As a company in business for 25 years, Human Inference absolutely sees the benefits of the internet. For our reasoning processes, based on natural language processing, we gather content and we classify this content on type, such as given names, family names, prefix, suffix, etc. (See also my blog post on the comparison of apples and oranges ….)

In the past this was done manually by, for example, investigating telephone books or manual research of census lists. But these were the ‘pioneer years’. What we see now is an enormous amount of content that can be gathered on the internet. It’s quite easy to find an internet page with 180 million records of person names. Great, so knowledge gathering is passé now?

Sorry to say; it has become far more complex now. The volume is much larger than in the past, but the quality is worse. Of course there are nuances in quality (e.g. the difference in content coming from social media versus the content from census lists), but in all these cases people working in data quality will absolutely agree with me, that even lists coming from the government contain false names and garbage.

Before we can discuss how to gather the right content from a large set, we need to have a common understanding on what is a right name. In discussions with linguists you can have long debates about this. Take my family name for example: according to the Dutch rules the family name “van Holland” is valid and known, and the family name “Van Holland” or “Vanholland” is invalid and unknown. In an international setting however, both names are absolutely valid and known (for example in Belgium or the United States) Similar situations might occur when people move between countries and patronyms (names derived from the father’s name) are used in one country, but not in another. Then you could find a name like Maria Romanov, who, in her country of origin, would be named Maria Romanova.

This phenomenon, which I call the ‘degenerations’ of names, has been going on ever since official administrations wanted to store names of individuals. That’s the reason why there are, for example, so many variants of the name Mohammed. Sometimes this cannot be prevented because there are characters in the original name which are not known at the registration office. An example would be the family name Sørensen (quite common in Scandinavia), which would become Sorensen, because of the unavailability if the letter ‘ø’. The moment a name is officially registered and used as such, the name becomes a valid name. This becomes even more complex once a name is written in a different writing set and has to be transcribed or transliterated.

We see here that right and wrong needs to be seen in context of region. We define a name as right in case the individual related to that name identifies her/himself with that name. So, in case I’m travelling to the US, I might still feel that “Vanholland” is wrong, but I can imagine that my relatives living in the US for many years now call themselves “Vanholland”. They identify with that name and in that case the name becomes “right”.

Back to the 180 million records. In order to filter these to get valid names we first perform so-called purification filters. That’s a huge set of different filters divided in specific sets like salutation filters, company related word filters, extraordinary sequence filters, strange character filters, capitalization filters, etc. Examples on what we filter here is for example the name “be@home” with the strange character filter but we keep the “d’Ancona”, or filter on “John Baker Inc.” with the legal forms filter in the company related words set. Nice examples in the extraordinary sequence set are always the words on our black list (e.g. “asdf” or “qwerty”) or repeating words (e.g., “abcabcabc”). After this purification we end up with 160 million records.

These 160 million are aggregated on family name and the frequency is counted. This causes a major reduction, because now we have a list of 14 million potential family names. If we compare these 14 million with our internal knowledge we immediately see the difficulty of dealing with names, i.e., the long tail. In general, the top 10% of the names in regions are responsible for over 90% of the population. The so called long tail, 90% of the remaining names, is responsible for less than 10% of that population. This means that this long tail of names has a very low frequency, as have most of the wrong names! So on frequency alone you cannot distinguish if a name is valid or not! For example we see that within these 14 million names, roughly 1.1 million are responsible of over 140 million persons in the original population! The remainder of 12.9 million family names still needs a lot of attention to verify if they belong to valid names. An example of such names with their respective frequency is given in the table below and immediately shows that it is not trivial to assume that all these names are valid:

Roughly 3 million of these names have a frequency of 1! They are very hard to validate automatically, but there are techniques we use which I will describe later in a separate blog.

In order to handle these volumes of records and to place records in the right validation buckets we use the DataCleaner . This is a very powerful and fast open source profiling toolset. With DataCleaner 2.0 we also use the validation steps and the fact that you can now make a complete hierarchy of these steps in a single template. This way, the results can be stored in new data stores, that can be examined later within the same tool!

We are not afraid of tackling large datasets to gather content. A large dataset will not provide you automatically with a large update in your knowledge, there is an enormous amount of pollution in such a set. We use DataCleaner for purification because it’s powerful and extremely flexible. We have defined a complete process to purify a given dataset and to automatically retrieve as many valid names as possible, which will then be added to our knowledge. Manual work will remain, but we can focus now more on linguistically and culturally challenging names, instead of spoiling most of our time on trivial ones. Purifying 180 million records is a hell of a job, but we are good at it!

A major bank in Dongguan (China) refused a potential customer because his name is Li Jun. Apparently, there were already over 300 bank accounts assigned to the name Li Jun. Not that this particular Li Jun was responsible for opening all these accounts, there were just too many men with exactly the same name. The bank states that the refusal is nothing personal, since nobody with the name Li Jun will be accepted as customer in the near future….. In the meanttime, Li Jun is taking legal action against the bank.

When I read this news article this morning, my first thoughts were that it was perhaps a hoax. It turns out , however, that the news fact is true. From a data quality point of view this strikes me as really strange. How does this particular bank manage its customer data? Are there no additional identifiers (address, date of birth, etc.) to determine that you are actually dealing with the customer you think you are dealing with? Imagine that every John Smith would have a hard time to open a bank account, to apply for a job or to buy a product via the web. Or Jenny Jones? Bob Johnson? When is a name too “common”? It is common misbelief that the complexity of ideographic characacters such as Mandarin Chinese makes it harder to identify. At Human Inference we carried out some pretty serious dedups of Chinese files and-taking into account that Mandarin Chinese is a tonal language and other priciples of fault-tolearnce apply- the duplicate identification was rather accurate.

It is all a matter of using an intelligent data matching method and knowing what kind of data one is working on. Every name can be identified; even “common” names.

This watchlist was created in 2004 from several other lists and at that time it consisted of about 68.000 entries. I use the word entries, because in the years after it became fuzzy if one record is the same as one individual. By the end of 2008 the list had grown to over 1,1 million entries. In 2008 after the American Civil Liberties Union (ACLU) mentioned that the list had passed the 1 million, the government came with an explanation. Although we have recorded over 1 million entries in the database, the net result is that these records correspond to about 400.000 individuals. Terrorist often use different and thus multiple identities, use several (falsified) passports etc. But adding entries with only the first initials and last name, while an entry of the full first names and last name already exists will result in unwanted side-effects.

We all know, as being interested in data quality and identity resolution, that J. Robinson will result into much more matches (hits) than James Robinson. Indeed the number of found matches will sky-rocket and have to be evaluated manually. Might this be the reason, that we see more and more security personnel on airports?

In the latest audit report of the U.S. Department of Justice about this watchlist one other problem was analyzed. While extensive procedures were made for nominating and adding suspects to the watchlist, there is no procedure for removing people from the list. Based on a sample of almost 70.000 entries and investigation of the individuals an astounding number of 35% omissions was found. People who had died were still on the list, people who were no longer investigated upon, cases which had been closed etc. So this watchlist is growing and growing. Resulting in screening personnel who ensnare many innocent travelers as suspected terrorists. And wasting their time and divert their energies from looking for true terrorists. It seems to me that FBI and TSC can benefit from better Data Governance, what do you think?

A survey of Human Inference in 2008 indicates that processes are the biggest experienced challenge in relation to Data Quality. However the subject that seems to be no problem is the budget. Human inference differentiates itself by interpretation of knowledge. So from this perspective I wonder how the respondents interpreted the word “processes”. Do they mean the processes within the value chain of their companies or do they actually mean the process of obtaining a budget for Data Quality? The latter would actually explain a lot.

HI Survey Results