How To Remove Duplicate Master Data With Open Documents

Most data migration specialists take great pains to ensure that duplicate Vendor Masters, Customer Masters and Material Masters are not created during SAP implementations, but are instead extended to the Sales Orgs, Plants and Company Codes as a firm rolls up acquisitions and expands. Sadly, many fly-by-night organizations and even Big 6 firms will take shortcuts which result in duplicate records created and immediately used in a production system.

This is the sort of thing that is easily identified and resolved after the first System Integration Test, but during a recent project, it was discovered that tens of thousands of duplicate Master Data records slipped by the system test, and, if there were any, the UAT. Naturally, this leads to great consternation as managing interactions with customers and vendors through multiple Customer Masters and Vendor Masters causes undue confusion and renders downstream reporting near useless.

After spending months identifying the duplicates by eye, it was time to develop a plan to sunset the Master Data that, by dint of being the duplicate of other data, did not need to exist in the system. For the purpose of this whitepaper, the duplicate Master Data record chosen to remain in the system is referred to as the survivor and the duplicate chosen to sunset (usually because of disuse or by being extended to fewer plants, Sales Orgs and Company Codes than the survivor) is referred to as the victim.

Master Data with no open transaction documents like Sales Orders, Purchase Orders, ARs, APs, Production Orders and inventory were considered low-hanging fruit. They were the fist to go as they would not be missed and their sudden absence would pose no risk to business continuity.

Those duplicate Master Data records with open transaction documents posed a problem as a window of time existed between the day that a duplicate was identified as a victim and the day that all of the associated open documents were closed and the duplicate could be sunset. A method was needed to make users in the business stop using victim duplicates and only use survivor Master Data when creating new documents.

After toying with the idea of adding a special character like an asterisk or tilde to the names of all victim Master Data and then training the thousands of users to self-police themselves such that all new documents would be created using the survivor Master Data, it was concluded that this simply wouldn't work as the business spans multiple continents and users are creatures of habit. Occam was right and two very simple solutions proved to be very effective.

Use a Popup Window to Make Users Self-Police

Adding simple config to cause a popup window to appear when a Master Data record is entered into a transaction can be used to direct users to stop using the victim. By including an explanation of why the victim must no longer be used and the number of the survivor, all new transactions will be applied to the survivor such that the open transactions associated with the victim can be fulfilled and closed allowing for the victim to be completely sunset with no impact to business continuity.

In the event that the victim is still keyed into new documents by a user who ignores the popup, that user can be notified and further instructed to stop using the victim and to use the survivor mentioned in the popup. This isn't usually a problem as users tend to self-police when the popup appears and this method was used with great success.

Just hide the victims . . .

During the creation of documents, most users search for Master Data using the name in the "Restrict Value Range" popup instead of memorizing thousands of 10-digit or 18-digit numbers. By adding a search parameter for the "Deletion flag" and defaulting it to blank, only survivor Master Data that have not been flagged for deletion will appear in search results and victim Master Data will be hidden from the search results.

With this filter in place, the victim Master Data can be flagged for deletion and not be offered to users as a choice during the creation of new documents. This also eliminates confusion after the victim Master Data record has been sunset; no service representative need call the credit department to understand why a victim has been blocked for sales if he only sees the survivor.

After some months, when all of the open documents are closed, the victim Master Data can be blocked for posting to further ensure that it is never used again. After some years, a cleanup will remove all of the Master Data records flagged for deletion, but all of the history associated with those records remains in the system until then and can be accessed at any time by putting an X in the deletion flag line or by removing the "equal" filter.

To add this defaulted deletion flag to the "Restrict Value Range" popup, launch the OMSH transaction and then call up the search help for the desired Master Data. In this example, the "Vendors (General)" search help screen is modified.

In this example, victim Vendor Masters flagged for deletion no longer appear in search results, but most open documents associated with them are still processed to finalization as SAP ignores the deletion flag. Incoming goods receipts and invoices are tied to open Purchase Orders and not to the victim Vendor Master so they are still processed as the come in using the MIGO and MIRO transactions.



Can't pay deleted vendors!

Despite the way that SAP mindlessly accepts invoices associated by proxy with the victim Vendor Master through open Purchase Orders, open APs are not processed during automated or manual check runs because the victim Vendor Master is flagged for deletion. Until all of the invoices are paid, a once-per-week process of sitting with the AP department and removing the deletion flag long enough for the proposal to generate payments will be required.

Depending on the volume of open APs, this task can be completed in minutes while sitting with the internal resource who does the check runs (usually the manager of the AP department). Scheduling this as a once-per-week event lasting just a few minutes yields the least amount of push back as the inconvenience is more than offset by simplifying the purchasing process and reporting.

The lesson learned

In this case, an ounce of prevention would have obviated a ton of cure. The data migration specialists who loaded these tens of thousands of records could have spent a week eyeing the data to disambiguate the incoming records against each other and against the records already in the production system.

The forty hours they saved by blindly loading duplicate Master Data records instead of extending existing records cost their client more than one thousand man hours. The behind-the-scenes cost to the business in the form of creative accounting and manual reporting can be measured but the embarrassing inability to control how the business interacts with its customers and vendors is costlier.

Bureaucracy alert!

Configuration changes will involve using transactions that are not normally accessible to most users and will require a transport, so be prepared to deal with the BASIS team or some other bureaucratic organization. Each company handles this differently; it is very likely that the team responsible is external to the organization and hidden behind several layers of bureaucracy to make reaching them very cumbersome.