Data Duplication

ghilton · Post by **ghilton** » Thu Jan 22, 2009 12:22 pm

Hello

I am running this query to check the email attibute for duplication -

IF EXISTS Contacts WHERE (Contacts.EmailAddress = ThisContacts.EmailAddress AND Contacts.ID <> ThisContacts.ID) Then
ThisContacts.Duplicate = 'Yes'

The problem is this is not trapping all the duplicated email addresses. I have just exported the data to excel to check that everytning is working fine and have found another 200 duplicated emails?

Can any one see why this would be happening from the above query!

Thanks

tford · Post by **tford** » Thu Jan 22, 2009 12:54 pm

Hi Graham,

You say you are "running this query", but the rule you showed is not a query. A query would involve a FIND action. ???

Tom

ghilton · Post by **ghilton** » Thu Jan 22, 2009 12:55 pm

Yep I stand corrected this is a rule. I have this on the contacts BO checking this attribute

tford · Post by **tford** » Thu Jan 22, 2009 1:00 pm

This rule would only run when BO instances are created or changed. I'm not sure when you added this rule, but it's possible that it just has not fired yet it you first added the instances, then later added the rule.

If that's possible, run a process with:

FIND ALL Contacts
UPDATE Contacts

Tom

ghilton · Post by **ghilton** » Thu Jan 22, 2009 1:06 pm

Yep have a process to do this already

tford · Post by **tford** » Thu Jan 22, 2009 1:24 pm

Graham,

Something has to be different than you described. I just did a quick test BSV copy/pasting your rule into a test bsv & it worked fine.

When creating the second instance of Contacts with a dup email address, AwareIM properly turned Contacts.Duplicate to 'Yes' for the newly added intance. Of course the first one would not have been marked with a Yes until you run the FIND and UPDATE.

Your rule would not detect and fix removing the Yes once a duplicate is resolved.

This rule solved that:
IF EXISTS Contacts WHERE (Contacts.EmailAddress = ThisContacts.EmailAddress AND Contacts.ID <> ThisContacts.ID) Then
Contacts.Duplicate = 'Yes' ELSE Contacts.Duplicate='No'

Note that in the bold area I also removed an unnecessary THIS.

Tom

ghilton · Post by **ghilton** » Thu Jan 22, 2009 2:03 pm

Thanks Tom

I am not sure whats going on as these have been imported recently but the odd few are not being marked as duplication=yes and they are clearly duplicates!

tford · Post by **tford** » Thu Jan 22, 2009 2:07 pm

So have you run your FIND / UPDATE process?

When you try to edit and save one that is know to be a duplicate, the logs should show you what's happening.

ghilton · Post by **ghilton** » Thu Jan 22, 2009 2:15 pm

I have just created another process as you suggest

FIND ALL Contacts
UPDATE Contacts

When I run this all I get is 'connection failure'

?????

tford · Post by **tford** » Thu Jan 22, 2009 2:20 pm

I'm confused. Earlier you said you had a process to do this already & now you say you created the process I suggested.

Not sure what's up with your connection failure. I've never experienced that. Might want to try rebooting???

Tom

ghilton · Post by **ghilton** » Thu Jan 22, 2009 2:26 pm

Tom

Sorry for the confusion I thought I would just create a Process from scratch to see if this would make any difference. The Rule I have is

FIND ALL Contacts
IF EXISTS Contacts WHERE (Contacts.EmailAddress = ThisContacts.EmailAddress AND Contacts.ID <> ThisContacts.ID) Then
ThisContacts.Duplicate = 'Yes'
Update Contacts

On this I decided to write a process as you described to see if there was any difference!

tford · Post by **tford** » Thu Jan 22, 2009 2:29 pm

Here are the steps:

1) put this rule on the Contacts BO ---- not in your update process:

IF EXISTS Contacts WHERE (Contacts.EmailAddress = ThisContacts.EmailAddress AND Contacts.ID <> ThisContacts.ID) Then
ThisContacts.Duplicate = 'Yes'

2) run a process that includes:

FIND ALL Contacts
UPDATE Contacts

Running #2 will cause all the rules for the Contacts BO to fire.

Tom

ghilton · Post by **ghilton** » Thu Jan 22, 2009 2:33 pm

Tom

I do have a rule on the BO which is as said in your last post. I also have a process so I can check this maunually if I need to.

The problem is I still have duplicates and I am not sure why?

tford · Post by **tford** » Thu Jan 22, 2009 2:39 pm

Graham,

There has to be something else going on in your bsv because I've tested this in a sample bsv with the exact rules stated above. Not sure what else to tell you.

When you tried my earlier suggestion to "try to edit and save one that is know to be a duplicate, the logs should show you what's happening", didn't the logs reveal what's going on?

Tom

ghilton · Post by **ghilton** » Thu Jan 22, 2009 2:44 pm

no the logs are not showing anything so I thought I would post this here!

Support do you have any input?