For over 25 years Human Inference has collected reference data, for instance on persons. Because of our reference set we immediately recognize that ‘Ask Me’ and ‘Any Body’ are fake names. People are using these either in test situations or to hide their actual names.

In the old days we only needed to test on ‘Test Test’, in more recent years we see great inventiveness on these fake names. A brief example can be seen in the following list.

In case you cannot rely on reference data and interpretation you need to provide a check list. Providing it is one thing, but since users tend to be really creative, maintaining it is essential.

In these 25 years we identified a move from ‘real fake names’ towards ‘real names used in a fake way’. In the USA, for example, we identified popular Hollywood names and names of politicians being used as fake names. Currently the usage of the name ‘George Bush’ is decreasing, whereas ‘Barack Obama’ is increasingly used. We recognize the false usage of these names because of the change in frequency figures of the given name and family name as well as the usage of the combination itself. Remarkable is that ‘Abraham Lincoln’ and ‘George Washington’ are quite steady.

Back to ‘Walter von Stolzing’. By now you might have guessed what is happening here. We recognized that in German speaking areas this name is also passing our threshold on validity. By googling the name you can see that Walter is actually a character in Wagner’s opera ‘Die Meistersinger von Nürnberg’ back from 1868!

Let’s see if in 100 years time people are still using ‘Darth Vader’, ‘Lord Rings’ or ‘Snoop Dogg’!

All the names used in this blog are ‘real’ names coming from a popular social media site. Please check our data cleansing products in case you need cleansing solutions.

Everyone in this world has a name. When we hear a name however, it is really hard to precisely know what it consists of and how the consisting parts should be written. A name might, for example, contain salutation, one or more titles, given names, initials, one or more family names, and additions. Here’s an example of a name with different name parts:

Mr Peter M Smith PhD

• Mr – Salutation, also called honorific, a polite way of addressing a person
• Peter – Given name. The name given to a person at birth. You have male and female names, and sometimes a name can be carried by both. But, in general, it is possible to derive the gender of the person from his/her name(s).
• M – Initial. An abbreviated form of a given name
• Smith – Family name
• PhD – Addition, in this case an academic title

This may appear easy, but due to all different naming conventions in the world, it is definitely not! At Human Inference, we have automated this process by creating a Firefox plugin that can help you interpret the various name parts and assign a gender to the name. It also finds the names which most closely resemble the ones you typed as input.

You can type the full name of a person, and that’s all you need to do. The plugin will make the most probable interpretation, based on the vast knowledge of names it has. It places the parts in the appropriate fields and displays the predicted gender. On top of that it will give you close alternatives for the names or for the way the input can be interpreted. These are shown as a list of suggestions when you right click on the input field. If you think that any of the other suggested interpretations is what you were looking for, you can click on it and it is displayed instead.

Summarizing, the plugin

• can correct the mistakes that you made writing someone’s name.
• can be used to segment the name correctly.
• can provide you with closely resembling suggestions.
• can predict the gender of the name.
• is free of charge!

The plugin is powered by our HIquality Name Worldwide product. Name Worldwide has a vast knowledge base of names along with its gender, the region in which it is found and the frequency of the name. It’ s powered by intelligent algorithms which interpret the name as soon as you type it. Apart from interpreting a name, Name Worldwide provides lots of other features like validation and suggestion. And you can easily power your web form with Name Worldwide by writing less than 5 lines of javascript code. Imagine someone filling in your web form, and his or her name and lots of similar names pop up before they are even completely typed in……

Enjoy using the plugin and please contact Human Inference for further information on data quality solutions.

The success of many business processes is linked directly to the quality of customer data. This is not only an obvious fact, but a recurring conclusion of many field studies: Incorrect, incomplete and inaccurate data will have a direct impact on your business succes rate. The symptomatology of this increase is established in inefficient marketing and sales processes, customer dissatisfaction, difficult cross- and upsell, unreliable analyses and many other disturbances in the day-to-day business of almost every organization dealing with customer, supplier and/or partner data.

In essence, it all comes down to knowing your data, in order to be able to trust your data. If you trust your data, you are definitely doing something right. So, how do you establish that trust? For this, you first have to answer a short, yet rather complex question: What is what in my database(s)? In other words: You have to identify and interpret the data you are working with .

A robust customer data identification solution intelligently interprets the details of both natural and legal persons. That process has to take account of the significance of words in a specific context, usage of company names, abbreviations, synonyms, acronyms, spelling mistakes, notation methods, standards and phonetic similarity of words. All in all, this is not a simple task; it more or less mimics the capabilities that humans show when interpreting data …

It is, however, the first step in a solid data quality strategy. This strategy should entail some sort of methodic, recursive approach. This makes sense, since data cleansing is basically a process of recurring steps. Initial cleansing should, for example, be combined with methods to prevent future pollution. In other words: Do not only fight the symptoms of bad quality, but eliminate the root causes and make sure your clean data will stay clean. Underneath you will find an illustration of such an approach:

Effective customer service, targeted cross- and upsell, cost decrease and creation of cutomer lifetime value are but a few goals that will be achieved by defining and deploying the right data quality strategy. So start to know your customer and learn to trust your data…..

High quality customer data have become the prerequisite for successful business decisions. In order to reach the intended data quality level, a lot of money is being invested in solutions for input control, file merging, data enrichment and duplicate identification. But do these investments guarantee high quality data and information? For example, are the data quality tools and processes equipped for the inevitable internationalization of our business community? Is a football always a football?

Natural language processing

Why do we know that William Jones International Logistics Ltd and W. Jones Int. Transport Co. are probably different notations for the same company? How do we determine that Leonard is a given name in Leonard Peters and a surname in Leonard & Peters? Without being all that aware of it, we are using methods such as pattern recognition, context analysis and other linguistic considerations. To answer the question ”what is what in customer data?” people will use their knowledge of language and culture to interpret the data they will encounter in daily life.

Correct, automated interpretation of customer data needs to imitate this natural language processing abilities. This requires knowlegde, containing relevant information on the components customer data consist of. Furthermore, a “grammar” is needed to take care of issues such as context rules, ambiguity checks, structure recognition, semantic associations and probability estimates. Using the knowledge and the grammar, the software solution decides what is the most probable signification of a word in a database record. This is the basis for all customer data quality processes. A quick look at the following example illustrates the power of this approach. We, as humans, immediately understand the signification ambiguity of “ART”. In addition, we also understand that automated interpretation represents a high level of complexity:

Art Johnson Sporting Goods

Art Gallery Johnson & Johnson

ART Ltd. Auto Rendition Technology

If we take a look at international busines initiatives, things become even more complex. Apparently, a lot of companies doing business abroad, often seem to forget that they are dealing with a large variety of languages, names, address conventions and other culturally embedded business rules and habits. For this post, I will limit myself to some focus points when dealing with names in international context.

The names Haddad, Hernández, Le Fèvre, Smid, Ferreiro, Schmidt, Kuznetsov en Kovács all mean “Smith” in different countries. That’s a factual observation, which is not necessarily helpful in solving complex data quality problems. There are, however, many aspects of the various national naming conventions, that represent a challenge for every company doing business across the border. Here are some examples:

Signification of name components

Due to divergent naming conventions, there is a great variety in storage, exchange, representation and signification of names. For example, the first name Joan is male in Spain and female in Belgium. Also, the representation will be exactly reverse. The form of address ‘Señor’ is the male equivalent of ‘Mevrouw’. In Spain: Señor Joan Martinez Fonseca Andrade. In Belgium: Mevrouw Vandenwalle, Joan.

Prefix sorting

A name like Van Buren would be sorted under ‘V’ in the US. In the Netherlands, for example, that name will always be found under the letter ‘B’. Additionally, the spelling (initial capital or not?) of prefixes differs per situation, per country.

Patronymics

The use of patronymics (names derived from the father’s first name) is highly country-specific. Whereas a Russian man whose father’s first name is Ivan, will add the patronymic Ivanovich to his family name, his sister will use Ivanovna: Sergei Ivanovich Golubev and Olga Ivanovna Golubeva. In Iceland, it is impossible to establish direct relation through analysis of family names. Here the patronymic serves as the family name itself: The son and daughter of Björn Thorgeirson will be called Nils Björnson and Anna Björnsdottir respectively.

Naturally, there are many more data and information quality aspects to consider when crossing the border. Think of multiple character sets, privacy issues, multi-lingualism and different currency and date notation. The examples given are meant to illustrate the following: Companies working with international data must invest in understanding customer data specifics. One size does not fit all!

A major bank in Dongguan (China) refused a potential customer because his name is Li Jun. Apparently, there were already over 300 bank accounts assigned to the name Li Jun. Not that this particular Li Jun was responsible for opening all these accounts, there were just too many men with exactly the same name. The bank states that the refusal is nothing personal, since nobody with the name Li Jun will be accepted as customer in the near future….. In the meanttime, Li Jun is taking legal action against the bank.

When I read this news article this morning, my first thoughts were that it was perhaps a hoax. It turns out , however, that the news fact is true. From a data quality point of view this strikes me as really strange. How does this particular bank manage its customer data? Are there no additional identifiers (address, date of birth, etc.) to determine that you are actually dealing with the customer you think you are dealing with? Imagine that every John Smith would have a hard time to open a bank account, to apply for a job or to buy a product via the web. Or Jenny Jones? Bob Johnson? When is a name too “common”? It is common misbelief that the complexity of ideographic characacters such as Mandarin Chinese makes it harder to identify. At Human Inference we carried out some pretty serious dedups of Chinese files and-taking into account that Mandarin Chinese is a tonal language and other priciples of fault-tolearnce apply- the duplicate identification was rather accurate.

It is all a matter of using an intelligent data matching method and knowing what kind of data one is working on. Every name can be identified; even “common” names.

This watchlist was created in 2004 from several other lists and at that time it consisted of about 68.000 entries. I use the word entries, because in the years after it became fuzzy if one record is the same as one individual. By the end of 2008 the list had grown to over 1,1 million entries. In 2008 after the American Civil Liberties Union (ACLU) mentioned that the list had passed the 1 million, the government came with an explanation. Although we have recorded over 1 million entries in the database, the net result is that these records correspond to about 400.000 individuals. Terrorist often use different and thus multiple identities, use several (falsified) passports etc. But adding entries with only the first initials and last name, while an entry of the full first names and last name already exists will result in unwanted side-effects.

We all know, as being interested in data quality and identity resolution, that J. Robinson will result into much more matches (hits) than James Robinson. Indeed the number of found matches will sky-rocket and have to be evaluated manually. Might this be the reason, that we see more and more security personnel on airports?

In the latest audit report of the U.S. Department of Justice about this watchlist one other problem was analyzed. While extensive procedures were made for nominating and adding suspects to the watchlist, there is no procedure for removing people from the list. Based on a sample of almost 70.000 entries and investigation of the individuals an astounding number of 35% omissions was found. People who had died were still on the list, people who were no longer investigated upon, cases which had been closed etc. So this watchlist is growing and growing. Resulting in screening personnel who ensnare many innocent travelers as suspected terrorists. And wasting their time and divert their energies from looking for true terrorists. It seems to me that FBI and TSC can benefit from better Data Governance, what do you think?

In Holland we already have three officially approved manners of naming streets an addresses. Nowadays we have the TNT Post standard, based on the very first publication of the postal code book, and afterwards corrected several times. This naming convention was the basis for the NEN 5825 standard (NEN is the dutch variant of ISO). But the true source of streetnames comes from the municipality and is called “Raadsbesluit”.Due to the different versions this can easily result in five different ways of spelling of the same street, as the example below shows:

What do you think, should we add a new “English” standard to existing standards. What are the pro’s and con’s for having English labels of the streetnames, please add your opinion in the comments. Should we also replace all signs and add the English label for the streetnames? For more information and history of Dutch street names I recommend the following site “alles over straatnamen”.

A survey of Human Inference in 2008 indicates that processes are the biggest experienced challenge in relation to Data Quality. However the subject that seems to be no problem is the budget. Human inference differentiates itself by interpretation of knowledge. So from this perspective I wonder how the respondents interpreted the word “processes”. Do they mean the processes within the value chain of their companies or do they actually mean the process of obtaining a budget for Data Quality? The latter would actually explain a lot.

HI Survey Results