Data Duplication

ghilton

Hello

I am running this query to check the email attibute for duplication -

IF EXISTS Contacts WHERE (Contacts.EmailAddress = ThisContacts.EmailAddress AND Contacts.ID <> ThisContacts.ID) Then
ThisContacts.Duplicate = 'Yes'

The problem is this is not trapping all the duplicated email addresses. I have just exported the data to excel to check that everytning is working fine and have found another 200 duplicated emails?

Can any one see why this would be happening from the above query!

Thanks

tford

Hi Graham,

You say you are "running this query", but the rule you showed is not a query. A query would involve a FIND action. ???

Tom

ghilton

Yep I stand corrected this is a rule. I have this on the contacts BO checking this attribute

tford

This rule would only run when BO instances are created or changed. I'm not sure when you added this rule, but it's possible that it just has not fired yet it you first added the instances, then later added the rule.

If that's possible, run a process with:

FIND ALL Contacts
UPDATE Contacts

Tom

ghilton

Yep have a process to do this already

tford

Graham,

Something has to be different than you described. I just did a quick test BSV copy/pasting your rule into a test bsv & it worked fine.

When creating the second instance of Contacts with a dup email address, AwareIM properly turned Contacts.Duplicate to 'Yes' for the newly added intance. Of course the first one would not have been marked with a Yes until you run the FIND and UPDATE.

Your rule would not detect and fix removing the Yes once a duplicate is resolved.

This rule solved that:
IF EXISTS Contacts WHERE (Contacts.EmailAddress = ThisContacts.EmailAddress AND Contacts.ID <> ThisContacts.ID) Then
Contacts.Duplicate = 'Yes' ELSE Contacts.Duplicate='No'

Note that in the bold area I also removed an unnecessary THIS.

Tom

ghilton

Thanks Tom

I am not sure whats going on as these have been imported recently but the odd few are not being marked as duplication=yes and they are clearly duplicates!

tford

So have you run your FIND / UPDATE process?

When you try to edit and save one that is know to be a duplicate, the logs should show you what's happening.

ghilton

I have just created another process as you suggest

FIND ALL Contacts
UPDATE Contacts

When I run this all I get is 'connection failure'

?????

tford

I'm confused. Earlier you said you had a process to do this already & now you say you created the process I suggested.

Not sure what's up with your connection failure. I've never experienced that. Might want to try rebooting???

Tom

ghilton

Tom

Sorry for the confusion I thought I would just create a Process from scratch to see if this would make any difference. The Rule I have is

FIND ALL Contacts
IF EXISTS Contacts WHERE (Contacts.EmailAddress = ThisContacts.EmailAddress AND Contacts.ID <> ThisContacts.ID) Then
ThisContacts.Duplicate = 'Yes'
Update Contacts

On this I decided to write a process as you described to see if there was any difference!

tford

Here are the steps:

1) put this rule on the Contacts BO ---- not in your update process:

IF EXISTS Contacts WHERE (Contacts.EmailAddress = ThisContacts.EmailAddress AND Contacts.ID <> ThisContacts.ID) Then
ThisContacts.Duplicate = 'Yes'

2) run a process that includes:

FIND ALL Contacts
UPDATE Contacts

Running #2 will cause all the rules for the Contacts BO to fire.

Tom

ghilton

Tom

I do have a rule on the BO which is as said in your last post. I also have a process so I can check this maunually if I need to.

The problem is I still have duplicates and I am not sure why?

tford

Graham,

There has to be something else going on in your bsv because I've tested this in a sample bsv with the exact rules stated above. Not sure what else to tell you.

When you tried my earlier suggestion to "try to edit and save one that is know to be a duplicate, the logs should show you what's happening", didn't the logs reveal what's going on?

Tom

ghilton

no the logs are not showing anything so I thought I would post this here!

Support do you have any input?

tford

Graham,

I'm sure support will need the contents of the logs & not just " logs are not showing anything". If they "don't show anything", that means the rules are not firing at all.

Tom