Automatic uploading of References to Lost Cousins

[Alexander Bisset]

FTAnalyzer starting with v7.3 (Windows) and v1.2 (Mac - out soon) is able to automatically upload your census records to Lost Cousins.

As Lost Cousins is based on census references you need to have recorded a census reference in the notes for the individual in the source for a census/residence fact or in the description of a fact. If you don't yet have census references in your file then you can easily add them by looking up the census record automatically from the Lost Cousins tab and finding the reference.

Recognising a census ref is a tricky business as there are so many possible formats. As such it's a work in progress and I am constantly adding new methods of recognition based on what users tell me they have recorded.

I created a document to explain how to use the new features. You can find the latest version on the FTAnalyzer website.

If you are new to using FTAnalyzer you can download the getting started guide. This guide tells you how to download and install the app and to load a GEDCOM file into it for analysis. It even explains what a GEDCOM file is if you haven't heard of one before.

 

[JamesB]

Automatic uploading is a wonderful addition! Thanks!

Yesterday I uploaded 451 people from my tree. It recognized that 8 were already present, but apparently missed 16 (according to LC) that I now have duplicates of. And now I have to add all those LC facts to my tree!
I believe I have another 400 to look up the census ref for.

Unbelievably I did not get a single cousin hit from all those additions

On another note, I installed 7.3.3.1 over what I had from a long time ago, and there's a start up error that won't go away:

Error loading previously submitted Lost Cousins data. SQL logic error
no such table: LostCousins​

 

[Alexander Bisset]

Automatic uploading is a wonderful addition! Thanks!

Yesterday I uploaded 451 people from my tree. It recognized that 8 were already present, but apparently missed 16 (according to LC) that I now have duplicates of. And now I have to add all those LC facts to my tree!
I believe I have another 400 to look up the census ref for.

Unbelievably I did not get a single cousin hit from all those additions

On another note, I installed 7.3.3.1 over what I had from a long time ago, and there's a start up error that won't go away:

Error loading previously submitted Lost Cousins data. SQL logic error
no such table: LostCousins​

Hi James

v7.3.3.2 fixes the startup error. The issue was a typo in the code to setup the Lost Cousins table. In future, with this fix, it will pickup from your Ancestors page you have entered them already and so you won't need to manually enter those LC facts.

I'd be extremely interested to see the 16 duplicates so that I can try to determine why they are duplicates and what got missed in the checking for duplicates. Previous discussions had suggested that the website would reject duplicates but that doesn't always seem to be the case. So I'm assuming the duplicates are extremely similar but not similar enough to have been identified as already present? Could you possibly send me the information to ftanalyzer@gmail.com

Note the chance of getting a hit depends on how distributed your census records are. If they are mainly the 1881 E&W census the chance is a lot higher as that's the original census that was available on the site and that's the original one that was used when Lost Cousins started up.

However all is not lost as more and more people use the automated uploads the number of records added should shoot up and the number of entries will eventually hit a critical mass where the chances of a match is practically guaranteed.

Note you did click the search button after the upload so the website checks for new matches?

 

[JamesB]

Hi James

v7.3.3.2 fixes the startup error. The issue was a typo in the code to setup the Lost Cousins table. In future, with this fix, it will pickup from your Ancestors page you have entered them already and so you won't need to manually enter those LC facts.

I'd be extremely interested to see the 16 duplicates so that I can try to determine why they are duplicates and what got missed in the checking for duplicates. Previous discussions had suggested that the website would reject duplicates but that doesn't always seem to be the case. So I'm assuming the duplicates are extremely similar but not similar enough to have been identified as already present? Could you possibly send me the information to ftanalyzer@gmail.com

Note the chance of getting a hit depends on how distributed your census records are. If they are mainly the 1881 E&W census the chance is a lot higher as that's the original census that was available on the site and that's the original one that was used when Lost Cousins started up.

However all is not lost as more and more people use the automated uploads the number of records added should shoot up and the number of entries will eventually hit a critical mass where the chances of a match is practically guaranteed.

Note you did click the search button after the upload so the website checks for new matches?

Yes, I did click the search button I see your point, as there were only 45 from 1881, with the majority from 1911. I will send you the details on my duplicates. As a happy circumstance from looking at one family with duplicates, I discovered that the boarder was actually another relative.

 

[peter]

there were only 45 from 1881, with the majority from 1911

If you check your Match Potential before and after an upload you'll be able to see how it has increased. Currently an entry from 1881 is nearly 4 times more likely to produce a new match than one from 1911, and nearly 3 times more likely than one from 1841.

Something else to bear in mind is that your entries will only match with your cousins' entries if the information you enter matches the census (the transcript in 1880/81, the handwritten census in other years). So use the grey arrows to check your new entries (sort them in date order).

 

[CathyR]

FTAnalyzer starting with v7.3 (Windows) and v1.2 (Mac - out soon) is able to automatically upload your census records to Lost Cousins.

Recognising a census ref is a tricky business as there are so many possible formats. As such it's a work in progress and I am constantly adding new methods of recognition based on what users tell me they have recorded.

Hi Alexander
I used FTA to upload all the 1911 census entries, for which I had cut & pasted the reference from the FMP site. It took seconds only, once I had realised how to link FTA & LC!! However, all my other census references are simply in the format RGx-xx-xx-xx or (HO107) & of course FTA doesn't recognise them.
Any chance of yet another new method of recognition (in your spare time!) ?
Cheers, Cathy

 

[Alexander Bisset]

Any chance of yet another new method of recognition (in your spare time!) ?

Yes can you perhaps email me a GEDCOM of a single individual who has these references

I'm thinking you mean it doesn't recognise references separated by a - but does recognise them separated by a /?

Downside is there is nothing to determine what order they are in so I will assume the same order as on the Lost Cousins page. If that's wrong I cannot do anything as there is nothing to tell me what number is what.

1841 Piece/Book/Folio/Page
1881 Piece/Folio/Page

 

[kirkstall]

Alexander

Wow!

However, I use Family Historian and it has not created the Lost Cousin event and so when I load it again it appears that none of the entries have been added.

Is there a way to automate this process? (I have over 1,000 to do)

 

[Tim]

Hi Kirkstall,

Where are you expecting to see the Lost Cousin fact? My understanding is that FTA creates one in a table that it uses, so FTA knows which ones that you've added via the automated process. These LC facts only ever exist in FTA. If you want to have them in your tree then you're going to need to manually create them in Family Historian.

 

[PhilGee]

I have done a quick test and it is possible to create a file and merge that with your existing tree - the merge worked for a "hand-crafted" GEDCOM file and Family Historian.

You can also export details of those "to be entered" for a census year to a spreadsheet .csv file when in the "lostcousins" tab of FTA, though this then needs slightly complex processing - which is the difficult part - as the surname needs to be between "/" characters and the "married (birth)" format needs changing to "birth".

However, an additional "Lost Cousins" tab in FTA to list just the "Ind Id" and "Name" as Forename(s) /birth surname/ could be easily processed in a spreadsheet Note: the "Name" is required by FH to get a match, otherwise new "people" are added with a new Id.

So, by having "Ind Id" in column A and "Name" in column B then adding the census year in cell C1 and formula ="0 @"&A1&"@ INDI*1 NAME "&B1&*1 EVEN*2 TYPE LostCousins*2 DATE"&$C$1 in cell D1, as many entries as needed for a single census can be created by copying D1 down the column to generate cells containing data such as 0 @I15@ INDI*1 NAME Robert Henry /Scott/*1 EVEN*2 TYPE LostCousins*2 DATE 1911
follow this by a cut&paste of a column D into a text editor and simple edit to replace "*" by a newline character then add a header and footer added to give (the blank lines are formatting only and are not required)

0 HEAD
1 SOUR FTM
2 VERS 22.0.1.1474
1 CHAR ANSI
1 GEDC
2 VERS 5.5.1
2 FORM LINEAGE-LINKED

0 @I15@ INDI
1 NAME Robert Henry /Scott/
1 EVEN
2 TYPE LostCousins
2 DATE 1911

0 TRLR

Phil

 

[Katie Bee]

However, I use Family Historian and it has not created the Lost Cousin event and so when I load it again it appears that none of the entries have been added.

Where are you expecting to see the Lost Cousin fact? My understanding is that FTA creates one in a table that it uses, so FTA knows which ones that you've added via the automated process. These LC facts only ever exist in FTA. If you want to have them in your tree then you're going to need to manually create them in Family Historian.

Like Kirkstall I had misunderstood and thought that FTA was going to create a Lost Cousins Fact for me!
Thanks Tim for making it clear that it does not.
I have just been trying out the uploading and had not checked until reading this post.
At least I only had a few Lost Cousin Facts to add.

FTA is a great program and I have just downloaded the guides to help me get the most out of it.
Thanks Alexander

 

[Katie Bee]

I had one duplicate in my uploads.
Same name and everything, but FTA used an age of 22 instead of 23 and I did not have the maiden name in the lost cousins entry that I had entered manually.
Birth year 1858, census year 1881.

 

[canadianbeth]

I must be entering my census data incorrectly into FTM. I just added a few that were not in Lost Cousins so I could check and when I activated FTA they did not move to LC. I added them using the Person tab, typed in 1841 (or 1911) census and then the piece and other numbers, as well as the website address. What have I done wrong?

ETA: I added the family for 1911 and immediately got three red check marks. They are connected to others already checked.

 

[jorghes]

I must be entering my census data incorrectly into FTM. I just added a few that were not in Lost Cousins so I could check and when I activated FTA they did not move to LC. I added them using the Person tab, typed in 1841 (or 1911) census and then the piece and other numbers, as well as the website address. What have I done wrong?

ETA: I added the family for 1911 and immediately got three red check marks. They are connected to others already checked.

I am not going to be able to give you the entire answer, but I think FTA uses the source citations to find its information - I have been editing source citations (in FTM and Ancestry) to add missing schedules for the 1911 census and it seems to have been working.

 

[Alexander Bisset]

To clarify FTAnalyzer if it uploads your records to Lost Cousins will store that info in a small database file. When you load it again it will read this database file and re-populate the Lost Cousins facts. So the Lost Cousins facts don't get written back to your family tree program (there are just too many variations of family tree program to deal with) but they are recognised every time you load your file again.

The one issue when this didn't happen was when the program was giving a SQL error on startup. It was trying and failing to create the table to store the LC facts in. So any uploads whilst that error was in effect would result in no LC facts. That said the next time you try to update it will read your LC ancestors page and create new LC facts so you can always re-run an update to force a re-creation of the missing facts now that bug is fixed.

 

[Alexander Bisset]

Wow!

However, I use Family Historian and it has not created the Lost Cousin event and so when I load it again it appears that none of the entries have been added.

Is there a way to automate this process? (I have over 1,000 to do)

It should do it automatically assuming you are on v7.3.4 or higher.

 

[Tim]

I must be entering my census data incorrectly into FTM. I just added a few that were not in Lost Cousins so I could check and when I activated FTA they did not move to LC. I added them using the Person tab, typed in 1841 (or 1911) census and then the piece and other numbers, as well as the website address. What have I done wrong?

Where are you adding the missing schedules? I found it easier to add them in Ancestry, much quicker.
You add it under Source Citation

 

[Alexander Bisset]

I must be entering my census data incorrectly into FTM. I just added a few that were not in Lost Cousins so I could check and when I activated FTA they did not move to LC. I added them using the Person tab, typed in 1841 (or 1911) census and then the piece and other numbers, as well as the website address. What have I done wrong?

ETA: I added the family for 1911 and immediately got three red check marks. They are connected to others already checked.

EXACTLY what format of census reference are you adding and where are you storing it?

FTAnalyzer uses pattern matching to look for census references and so to work out whether there is a match or not it scans through 50+ different patterns to find a match. Although people are showing me new formats all the time and they are getting added. This is why I am saying what is the EXACT format so I can check the patterns.

 

[trebor]

Thanks Alexander - this has made the process of updating Lost Cousins so easy. Plus a new match - just waiting for a reply now.

However I encountered problems when I tried to update my wife's entry.
I changed the root person and the log in details in FTA and closed down the web page in Chrome for Lost Cousins. Possibly should have also logged out but I didn't!
Ran the update and it added all of the entries to my account.
Any suggestions on any easy way to correct this please?
I know I can delete the wrong entries in my account in Lost Cousins but there are over 200 of them and I can only see a way to delete them individually.
To update them to my wife's account I guess I have to delete the database referred to in another post that tells FTA what has been uploaded but where is this, what is it called and will it also delete the data for my account?

If anyone can break something it has to be me!!!

 

[peter]

Thanks Alexander - this has made the process of updating Lost Cousins so easy. Plus a new match - just waiting for a reply now.

However I encountered problems when I tried to update my wife's entry.
I changed the root person and the log in details in FTA and closed down the web page in Chrome for Lost Cousins. Possibly should have also logged out but I didn't!
Ran the update and it added all of the entries to my account.
Any suggestions on any easy way to correct this please?
I know I can delete the wrong entries in my account in Lost Cousins but there are over 200 of them and I can only see a way to delete them individually.
To update them to my wife's account I guess I have to delete the database referred to in another post that tells FTA what has been uploaded but where is this, what is it called and will it also delete the data for my account?

If anyone can break something it has to be me!!!

If you create a referral listing all of the relatives that need to be moved I can transfer them for you (but it will take about half an hour of my time so please, everyone else, don't make the same mistake). Don't include in the referral relatives that have matched - these will need to be deleted and re-entered manually.