Automatic uploading of References to Lost Cousins

[Alexander Bisset]

v7.3.7 is now available and should be more robust at checking ages and prompting users to check their entries after upload.

 

[Ian]

Absolutely brilliant Alexander, FTA found an additional 147 people to enter on LC, 8 errors, all to do with mistaken census ages - all "0m" (zero months) - where my information were all correct as per the census. No matches, but hey they're uploaded. Many thanks Alexander for FTA and your guide pdf.

 

[Alexander Bisset]

Absolutely brilliant Alexander, FTA found an additional 147 people to enter on LC, 8 errors, all to do with mistaken census ages - all "0m" (zero months) - where my information were all correct as per the census. No matches, but hey they're uploaded. Many thanks Alexander for FTA and your guide pdf.

I discovered a bug on the v7.3.7.0 fixed in v7.3.7.1 it seems that if you had an exact birth year it was showing age as ok but erroneously recording the age by treating the person as an infant (ages 0m-12m)

Note that this version has only been out for 13 hours and only 172 records were uploaded in that time. This has affect 6 people who uploaded 1, 4, 5, 2, 142 and 18 (me). So if the 3 people who uploaded 1, 4, 5 individuals had exact births they will have had the same issue. But that's a max of 10 records.

I've gone through mine now and all fixed but I'd recommend you check yours as it is likely you have erroneous ages even though you have exact births thus it shows the right year.

I've put out the emergency patch and pulled the previous one, plus posted a message for manual updaters to urge immediate updates.

 

[kirkstall]

I have loaded over 1,000 entries, all for 1841 and 1911 as I already had 1881 up to date.

It has thrown up many errors, many which can only be rectified manually (for instance changes in the spelling of a surname). All these errors are now on LC. I am doing my best to correct them but with over 1,00 to do it will take some time.

One thought I had is that if FTA put the matches into a holding file these could be checked and then uploaded to LC after they had been checked. Not only would this keep LC free of erroneous records but it should speed up the checking process. It would also be useful if the process could be split into small batches.

 

[Katie Bee]

I have decided that adding relatives to Lost Cousins is easier to do manually, especially for small numbers.
FTA uploaded 13 entries and I had one duplicate, 4 names that were different on the census and 8 years/ages that were different.
There were only 2 entries that did not have to be corrected.
It seems like a great idea, but difficult to program to get all the correct information added to Lost Cousins.

Thanks, Peter, for the email to remind me to check the entries.

 

[jorghes]

I have loaded over 1,000 entries, all for 1841 and 1911 as I already had 1881 up to date.

It has thrown up many errors, many which can only be rectified manually (for instance changes in the spelling of a surname). All these errors are now on LC. I am doing my best to correct them but with over 1,00 to do it will take some time.

One thought I had is that if FTA put the matches into a holding file these could be checked and then uploaded to LC after they had been checked. Not only would this keep LC free of erroneous records but it should speed up the checking process. It would also be useful if the process could be split into small batches.

I know your pain... I'm slowly moving through my own over 1000 entries...

 

[Alexander Bisset]

I have decided that adding relatives to Lost Cousins is easier to do manually, especially for small numbers.
FTA uploaded 13 entries and I had one duplicate, 4 names that were different on the census and 8 years/ages that were different.
There were only 2 entries that did not have to be corrected.
It seems like a great idea, but difficult to program to get all the correct information added to Lost Cousins.

Thanks, Peter, for the email to remind me to check the entries.

Examples would be exceptionally helpful to understand and eliminate issues. It’s also extremely useful to know what version you uploaded them with.

Duplicates simply should NOT be possible but whilst several people have told me they have seen duplicates, annoyingly NOT ONE person has actually provided an example. I CANNOT fix what I cannot see. It is exceptionally frustrating. I want this to work and I want it to be as seamless as possible. If I don’t get information on what is going wrong then it will never get fixed.

Note this is absolutely NOT directed at you Katie Bee. It’s a plea from the heart to all testers to give feedback.

 

[kirkstall]

I am not sure what people mean by duplicates but I have found two entries which have been added when they were already there, ie I had added them previously, FTA has not added them twice.
I am not sure what to send you. The GEDCOM of the entries will not show anything and I don't know how to copy the entries in LC. I will describe what has happened.
The 1911 census shows the forenames as Fred and Rose Han. I had originally entered them like that. The 'real' names are Frederick and Rosanna. FTA has added two entries with the 'real' names.
I will not delete the duplicates until I hear from you.

 

[peter]

The 1911 census shows the forenames as Fred and Rose Han. I had originally entered them like that. The 'real' names are Frederick and Rosanna. FTA has added two entries with the 'real' names.
I will not delete the duplicates until I hear from you.

Those are not duplicates as far as LostCousins is concerned (ie Fred will only match with Fred, Frederick will only match with Frederick, though you might see a near match warning against both entries). I'm not sure about FTA.

Incidentally, Fred is not always an abbreviation of Frederick - my great uncle is Fred on his birth certificate. However my great-grandfather forgot that when he filled out the 1911 Census schedule, showing his son as Frederick.

 

[PhilGee]

I have decided that adding relatives to Lost Cousins is easier to do manually, especially for small numbers.
FTA uploaded 13 entries and I had one duplicate, 4 names that were different on the census and 8 years/ages that were different.
There were only 2 entries that did not have to be corrected.
It seems like a great idea, but difficult to program to get all the correct information added to Lost Cousins.

As I see it, the problem here is that the GEDCOM names are decided by the user based on birth/baptism/common use as are the birth dates. These frequently differ from the census data, so how is FTA to "know" what should be inserted - it would be a mammoth effort to even try to associate "birth names" with "census names"*?

As it happens, I add the census name and age to my census data in the GEDCOM - just because my "pre-GEDCOM database" has been set up to hold this data.

Phil

* Try matching "Elizabeth Alice Mary Maud" with "Kate Elizabeth"

 

[Katie Bee]

Note this is absolutely NOT directed at you Katie Bee. It’s a plea from the heart to all testers to give feedback.

No offence taken, Alexander, you are doing a great job. FTA is a wonderful product, it just takes a bit of getting used to at times.

My problem was the same as Kirkstalls.
I already had an entry in Lost Cousins for the person in the 1881 census and a Lost Cousins Fact for 1881 in my Gedcom.
FTA uploaded it again and at first I could only see a difference in age 22 via FTA rather than 23 on the census.
Then when Peter emailed me to check the records uploaded I realised that my entry had a different forename.
The census read Elisabeth whereas she is Elizabeth in my Gedcom.
As PhilGee says it is difficult for FTA to check all names, but my record had a Lost Cousins Fact, so could FTA check if the record has a Lost Cousins Fact and then not upload Lost Cousins, but issue a warning to check if the record should be uploaded.

I also had an issue with the ages recorded by FTA when it uploaded to Lost Cousins. But I am just going out and don't have time to continue.

Edit:- Sorry forgot to give the time the entry was uploaded to Lost Cousins as I do not know the version of FTA that I used.
2019-01-30 21.08.34

 

[kirkstall]

Those are not duplicates as far as LostCousins is concerned (ie Fred will only match with Fred, Frederick will only match with Frederick, though you might see a near match warning against both entries). I'm not sure about FTA.

But they are duplicates as far as I am concerned as they are the same person.

I'm forgot to say which version of FTA I used. I loaded my records twice. The second time was 7.3.5.? I think

 

[Katie Bee]

Then when Peter emailed me to check the records uploaded I realised that my entry had a different forename.
The census read Elisabeth whereas she is Elizabeth in my Gedcom.

Sorry I have given you the wrong information.
I deleted my entry from Lost Cousins, so I was going on memory - the wrong thing to do!
I had a list of my entries at Lost Cousins before I started uploading any using FTA and I had entered her as Elizabeth.
So I do not know why FTA uploaded her again.
Same name, but FTA used an age of 22 instead of 23 and added her maiden name to the entry, my LC entry did not have her maiden name noted in the additional information. In the Gedcom - Birth year was 1858, census year 1881.

 

[Bryman]

But they are duplicates as far as I am concerned as they are the same person.

Then it is your immediate responsibility to remove the erroneous entry from your My Ancestors page before anyone can match with the false entry.

Peter has created a system which does not allow real duplicates (ie IDENTICAL values for reference, name and DoB). If you (or FTA on your behalf) enter different values for the same person then LC will assume that is a different person, whatever you may believe.

 

[Alexander Bisset]

I am not sure what people mean by duplicates but I have found two entries which have been added when they were already there, ie I had added them previously, FTA has not added them twice.
I am not sure what to send you. The GEDCOM of the entries will not show anything and I don't know how to copy the entries in LC. I will describe what has happened.
The 1911 census shows the forenames as Fred and Rose Han. I had originally entered them like that. The 'real' names are Frederick and Rosanna. FTA has added two entries with the 'real' names.
I will not delete the duplicates until I hear from you.

Ah yes as Peter says these aren't treated as duplicates as such by the website. What FTAnalyzer attempts to do is to use fuzzy matching on the name and compares what's on the website with what's in the GEDCOM. If it gets a match is says already uploaded if not it goes ahead. Note that it also matches on the age and has a 5 year tolerance so Katie's example confuses me as it should have seen Elizabeth X aged 22 and say yup that must be the same Elizabeth. I suspect something else was different however without actually having the full details of the two that are duplicated I can't be sure.

I will try to do some tests tomorrow as I don't know for sure without running some tests how Fred/Frederick pans out for fuzzy matching. I use a system provided fuzzy matching called Double Metaphones which is meant to be very good with names.

 

[Alexander Bisset]

But they are duplicates as far as I am concerned as they are the same person.

I'm forgot to say which version of FTA I used. I loaded my records twice. The second time was 7.3.5.? I think

Could you send me a screenshot by email of the details page for an example duplicate so I can properly test why it wasn't identified as a duplicate.

 

[jorghes]

Hey Alexander - I had another try with the PC version 7.3.7.4 and it came up with this error:
Failed to save Lost Cousins record in database error was: SQL logic error table LostCousins has no column named FullNameRecord

For every record, although it added them all to LC. It uploaded the Norse gentlemen with the strike through his 'o', but deleted the letter altogether, so he came up on LostCousins as "Jrgenson" and has seemingly duplicated all the information into the "correct..." sections.

 

[Alexander Bisset]

Hmm ok yes stripping characters is meant to ensure that only valid Lost Cousins letters get posted but in this case the "invalid" characters are needed. The problem is with the character it would fail to upload to Lost Cousins. Without it the name is wrong.

Peter should the letter be transposed to an O?

I've found an issue with the duplicate checking it was checking a person's surname against what was entered in the census and of course this was failing for married women hence it could add another version next time round if no Lost Cousins flag was created.

 

[canadianbeth]

I just loaded FTA again and updated to LC. Now, when I am checking my entries I see a little icon stating updated by FTAnalyzer beside some. However, it is beside one name in 1881 (600/124/4) that is already listed in a different entry (600/122/14), and when I click on the little arrow I get nothing from FTM. I have another in 1841 that also seems to be listed in a different entry (658/12/45/10) instead of 658/12/46/10 and the age is incorrect. That one also comes up empty in FTM and I cannot find him on Ancestry with those #s either.

However, I see one corrected in the U.S. 1880, where you added a person who was already listed, but I had him and his family with the incorrect page #. Thank you for that.

 

[jorghes]

I had an issue with my FTM file, and couldn't sync it, so I lost all my LostCousins facts on FTM... just re-synced it, and I'm slowly re-adding in my LC facts and deleting all the duplicates that have been added... it's going to be a LOOOONG process.