FTAnalyzer now searches census records

[Alexander Bisset]

The Lost Cousins Census Report in FTAnalyzer now has a new feature. The columns that are red indicate the census years where you have yet to find the person on a census.

The new feature in v1.5.4.0 and above is if you click on that red box it now opens your browser at a search results page on Ancestry. NB. this will require you to be signed into Ancestry as sadly it doesn't pay your subscription for you. It will currently only search UK Census records and so requires a subscription that includes searching UK census records.

I will add FindMyPast and FreeCen searches soon.

 

[Alexander Bisset]

Initial work on FreeCen completed. It needs a bit of work as currently it will always search ALL counties for birth rather than specific. I need to find a simple means to translate the county name in a person's file to match the county names on FreeCen's search page. I'll tweak that before uploading fix in v1.5.4.1

 

[peter]

Brilliant!

 

[Alexander Bisset]

Peter are the FreeCen results format for say 1881 sufficient for the LC website data entry? If they are then that might be the easiest first step to asking for permission to parse a FreeCen results page.

Piece: SCT1851/261 Place: Glenbervie -Kincardineshire Enumeration District: 5
Civil Parish: Glenbervie Ecclesiastical Parish, Village or Island: -
Folio: 195 Page: 25 Schedule: 117
Address: Drumlithie
 
          Surname        First name(s)        Rel        Status        Sex        Age        Occupation        Where Born        Remarks   
          BEATTIE        James        Head        M        M        35        Hand Loom Weaver Linen          Kincardineshire - Kinneff             
          BEATTIE        Christian        Wife        M        F        31                  Kincardineshire - Glenbervie             
          BEATTIE        Andrew        Son        -        M        10                  Kincardineshire - Glenbervie             
          BEATTIE        George        Son        -        M        8                  Kincardineshire - Glenbervie             
          BEATTIE        Jessie        Dau        -        F        7        Scholar          Kincardineshire - Glenbervie             
          BEATTIE        Mary        Dau        -        F        1                  Kincardineshire - Glenbervie

If that was possible we could soon have a situation where the program would show a user a missing 1881 census entry, they click on the entry to view the 1881 census and if the results are correct for their ancestors then the user could paste that URL back into something that posts the data to their My Ancestors page automatically.

What it couldn't of course do is type the result into their favourite family history program as there are so many of them out there it would be a nightmare to support.

 

[Alexander Bisset]

I've released v1.5.6.0 now that will automate searching Ancestry, FindMyPast or FreeCen census records for you.

To use this feature:

1. export your GEDCOM from your family history program,
2. run FTAnalyzer and load your file.
3. run the census report for a year you are interested in I suggest trying 1881 census first as that is the best year to use for Lost Cousins entries.
4. The census report will show you everyone that you have not yet found on a census for that year. (NB. this looks for a CENS or RESI fact in your GEDCOM if you only store a census as a note then your family history program won't record it as a census fact and thus FTAnalyzer won't know you have recorded something against that person)
5. The grey/white bands on the census report group together families; the ones in bold are ones the program knows are alive on that census (eg: they get married or die after the census date); the ones in red are your direct ancestors; NB, the ones in red AND bold are direct ancestors that are known to be alive on that census and thus really ought to be found.
6. Select your preferred search from the menu at the top, NB. you need a subscription with Ancestry or FindMyPast otherwise the search will fail.
7. Open your browser and login to your preferred search site. (The program will only be able to display a search result if you have logged in first)
8. Double click on the person you want to search for and the program will open the search page having entered all the search terms for that person for you.
9. Check the results to find your ancestor as you would have done if you'd typed the search terms manually
10. Record the results in your family history program and on Lost Cousins.

NB. if you find any issues with how it searches particularly if it isn't recognising a county name, please let me know and I'll attempt to fix the issue for a future version.

 

[trebor]

I've released v1.5.6.0 now that will automate searching Ancestry, FindMyPast or FreeCen census records for you.

To use this feature:

1. export your GEDCOM from your family history program,
2. run FTAnalyzer and load your file.
3. run the census report for a year you are interested in I suggest trying 1881 census first as that is the best year to use for Lost Cousins entries.
4. The census report will show you everyone that you have not yet found on a census for that year. (NB. this looks for a CENS or RESI fact in your GEDCOM if you only store a census as a note then your family history program won't record it as a census fact and thus FTAnalyzer won't know you have recorded something against that person)
5. The grey/white bands on the census report group together families; the ones in bold are ones the program knows are alive on that census (eg: they get married or die after the census date); the ones in red are your direct ancestors; NB, the ones in red AND bold are direct ancestors that are known to be alive on that census and thus really ought to be found.
6. Select your preferred search from the menu at the top, NB. you need a subscription with Ancestry or FindMyPast otherwise the search will fail.
7. Open your browser and login to your preferred search site. (The program will only be able to display a search result if you have logged in first)
8. Double click on the person you want to search for and the program will open the search page having entered all the search terms for that person for you.
9. Check the results to find your ancestor as you would have done if you'd typed the search terms manually
10. Record the results in your family history program and on Lost Cousins.

NB. if you find any issues with how it searches particularly if it isn't recognising a county name, please let me know and I'll attempt to fix the issue for a future version.

May I slightly modify your instructions to ensure people are using the right section of the program - the missing bits caused me confusion at first.

1. export your GEDCOM from your family history program,
2. run FTAnalyzer and load your file.
3. Select the Census tab
4. Select the Relationship Types to either restrict or extend your search.
5. If you wish to limit the search to a specific family enter the family name in the Surname window
6. Modify the age limitation if required to exclude older individuals or not.
7. Select the Census Date you are interested in - I suggest trying 1881 census first as that is the best year to use for Lost Cousins entries.
8. Click on Show Results
9. The census report will show you everyone that you have not yet found on a census for that year. (NB. this looks for a CENS or RESI fact in your GEDCOM if you only store a census as a note then your family history program won't record it as a census fact and thus FTAnalyzer won't know you have recorded something against that person)
10. The grey/white bands on the census report group together families; the ones in bold are ones the program knows are alive on that census (eg: they get married or die after the census date); the ones in red are your direct ancestors; NB, the ones in red AND bold are direct ancestors that are known to be alive on that census and thus really ought to be found.
11. Select your preferred search from the menu at the top, NB. you need a subscription with Ancestry or FindMyPast otherwise the search will fail.
12. Open your browser and login to your preferred search site. (The program will only be able to display a search result if you have logged in first)
13. Double click on the person you want to search for and the program will open the search page having entered all the search terms for that person for you.
14. Check the results to find your ancestor as you would have done if you'd typed the search terms manually
15. Record the results in your family history program and on Lost Cousins.

 

[Alexander Bisset]

Excellent thanks. It is very helpful to have someone review and improve the somewhat sparse documentation. I really ought to write more

 

[trebor]

Your walkthrough certainly helped me understand the program better.
My earlier attempts to use it were limited by a blinkered approach and only really looking at the Lost Cousins tab.
With this new understanding I may be able to add more guides - if you are happy with this I will get you to check them before releasing them to the world.

 

[Alexander Bisset]

Sounds good to me. An overview of the tabs and how I envisioned them may help. A starter for 10 as it were...

The Load tab is fairly self explanatory and simply shows the statistics of how many families, sources and individuals are loaded and how they break down. This is a double check against loading the correct file.

The individuals tab isn't that useful other than to confirm what is loaded although the columns can be sorted by clicking on them so you can sort on dates or locations etc which might be of limited interest in smaller trees.

The families tab shows all the families listed in the file again limited usefulness although again it can be sorted so may prove useful to some.

The locations tab is the first of the useful tabs (as I see it) it shows countries, regions, parishes and addresses. The idea being you can see from the countries tab which countries it has recognised and which are wrong. eg: in my latest file I just loaded I have a country called 22 Martin Lane, which means for something I've loaded I've neglected to add a proper full address. However I can dig down and double click on that "country" to see which individual and family is recorded at that invalid location. I can the go to my family history program and fix the error. I also seem to have a country of Agricultural Labourer which of course means I've recorded an occupation description in the place box by mistake. So again I can double click to see who it is and fix it in the family history program. By doing this you can tidy up your errors for locations.

Similarly for Regions you can check and see if for example you might have a England, Lincolnshire and an England, Linconshire clearly one is a typo so double click on the erroneous one and see who it is and fix it in your family history program (hereafter referred to as FHP). Once you fix these sorts of errors re-exporting a GEDCOM and reloading will show you if there are any other errors.

The locations tab also has a couple of buttons so it will search on Google Maps and/or Bing Maps to try to find the location. NB. If you never bother entering counties or countries in your tree then the location search on the maps is going to be very poor.

The occupations tab similarly shows any errors in occupations. I had several Ag Labs, some Agricultural Labs some Agri Labs, and some Agricultural Labourers. Now I try to standardise and this report helped condense the descriptions to a more uniform format. It also means I can look at Agricultural Labourer (my chosen format) and see that I have 82 of them and double click on that and see the list of them. Which may be interesting especially for the more unusual occupations.

The data errors tab shows common problems all typically to do with date recording especially for unlikely or impossible feats. eg: being married after they died. This report aims to help you eliminate these date errors which can sometimes be at the root of a brick wall when you've inadvertently dismissed some evidence as it doesn't fit only because you mis-typed a date at some point. At any rate eliminating obvious errors in your file is typically a good thing. It may also make you re-look at some sources when the error is made obvious like this.

The "Loose" deaths tab is an interesting concoction of my own. I've not seen the concept elsewhere, so bear with me whilst I explain. The concept is that by recording certain facts about someone you know which dates they must have been alive on. Similarly you know that they will have died before age 110. So the loose deaths attempts to warn of dates you have in your file that could be "tightened". The idea being that if you narrow down a death range for someone then it makes it easier to search. eg: if you have noted that a woman must be alive because they marry in 1878 and you note that their husband is a widower on the 1901 census then you know their death range is 1878-1901.

My theory is that if you enter that as a death date ie: the range "BET 1878-1901" then you tied down where to look for a death record. Now assume you had done that but hadn't then taken it another step and noted that they had 6 children the last born in Jun 1895. The report of Loose Deaths would say the range should be Jun 1895-1901 as the woman must have been alive at the date of birth to have given birth to the child.

If you follow this logic and "tighten" by updating the death record in your FHP to read "BET Jun 1895-1901" then the program will also know that the woman MUST have been alive on the 1881 and 1891 census and will highlight that fact in bold on the census report to make it more obvious that she is definitely there to find.

The census tab this has been covered above it shows who you have yet to find a census entry for and automates the filling out of search forms on the most popular search sites. NB. again if you are lazy about entering counties and/or countries in your tree then the automated searching is going to fail most of the time. As it can't work out how to narrow down your searches to the right country or county.


The Lost Cousins tab shows you people you have found on a census but have yet to enter onto Lost Cousins. Note this uses a Custom Fact called "Lost Cousins" you would need to add to your FHP and record against those you have entered. Sadly the program cannot magically work out who you have already entered you need to mark those it displays as needing entered as already on the Lost Cousins site. Tim's colour coded Census report shows your ancestors status for each census year. The idea of the Lost Cousins tab is essentially that it should always be blank indicating that everyone for whom you have a census entry is already entered into the Lost Cousins website. Similarly the colour coded report should ideally be all green!!

The Tree Tops tab this shows you everyone in your tree who has no parents. The idea is that we all get distracted and tend to concentrate on particular favoured branches often on male lines because of an attachment to a surname or because its often just easier (especially with the English records and no maiden names). The thought is that using this report to see the tops of our tree it might prompt further investigations and lead to pushing back some neglected branches. How often does a new source of data come along and we instantly look for our favoured line and never quite get round to the other lines. This report's intention is to help provide inspiration to search the more neglected lines in your tree. eg: by noticing that hmm that marriage ought to be in statutory I wonder why I never looked for it before? etc.

The War Dead tab shows those unfortunate generations of our male ancestors unlucky enough to have been born at the time that they may have served and possibly died in either World War. Since there are specialist resources dedicated to researching the wars having a list of possible male ancestors who may appear in enlistment/medal/memorial/etc lists was thought helpful.

Hopefully that gives some inspiration.

 

[trebor]

Thanks Alexander - that will certainly help a lot. Not only does it give inspiration but also shows the size of the challenge I have set myself.
I will start with simple guides for each tab similar to above. However if there is a demand for more detail with pictures / screengrabs I will try to do those so that they can be included with the program download.
If anyone prefers the detailed format please let me know.

 

[Alexander Bisset]

One of the nice things about Codeplex (the site that the program is hosted on) is that you can create page after page of documentation there and it can be edited and maintained separately from the application indeed by separate people if required. The application can then simply link to the online documentation saving having to have a help system built in. ie: the help can simply link to the online guides.

What would be useful therefore is if you manage to get some workable documentation, then we can upload it to the site and if you PM me your login name on Codeplex (it's a free registration) I can grant you editor rights to the documentation. Using this method means that updates/clarifications/edits to the documentation don't require releasing a new version and people having to download that it. The latest documentation would always be available online and I'd simply add a view help to the help menu and possibly context sensitive help too so that F1 pressed on a page would take the user to the relevant documentation web page.

 

[Tim]

Hi Alexander,

Thanks for post #9, that clears up a few things. I notice on the Census Tab you also have the "View Lost Cousins census Report", which appears to give the same result as the button on the Lost Cousins Tab. I was just wondering, on the Census Tab, you can filter by surname, can that be also utilised for the report on the Census tab? i.e. a Lost Cousins census report on just one surname?

Also, I used GenoPro, which is the best visualisation FHP I've seen, but it also allowed me to copy the data into Excel. I could then do multiple filters to help me narrow down areas of the tree that were a bit sparse.

From some of the tabs, e.g. the Indiviuals, would it be possible to do an export or enable users to highlight the cells they want to copy? As you have imported the whole gedcom, maybe there are other (or all) data facts that could also be included in the data export? I'm a data man, so having all the data from the gedcom in excel would be excellent, and if we find some really helpful filter settings, these could then maybe be turned into standard reports for users who don't want to use the excel option?

What do you think?

 

[peter]

Great to see that the documentation is coming along nicely.

Another aspect is the question of compatibility with various family tree programs - it would be great to be able to identify any potential problems up front (like the problem I had because I didn't start my tree with myself) so that users know what to expect and what they can do to solve any issues that they run into.

Once we've got that sorted, and if Alexander agrees, I'd like to write to a sample of the members who haven't entered any relatives at all on their My Ancestors page to see whether FTAnalyzer might be the catalyst that gets them started.

 

[Tim]

Once we've got that sorted, and if Alexander agrees, I'd like to write to a sample of the members who haven't entered any relatives at all on their My Ancestors page to see whether FTAnalyzer might be the catalyst that gets them started.

That sounds like a great idea.

 

[Alexander Bisset]

What I'd like to do is to use the documentation features at Codeplex. This allows creation of multiple pages and of course linking between them. So we could easily have a page devoted to each tab, a page outlining the general purpose, a page on compatibility with examples for each FHP, etc. As well as tips and tricks to get the most out of it. Ideally a few pages aimed at different audiences, someone using the program for the first time eg: Peter's suggestion of selected LC members; and perhaps one for those more advanced users too. So yes I'd be very happy if a "how to enter your first LC ancestors" page was created.

The Codeplex system allows me to grant registered Codeplex users (its entirely free) rights to edit pages on the project site. There is also an issues list there that is ideally the best place to record the list of issues and suggestions as well as bugs and complaints so that they are all sitting in the one to do list.

So ideally if some people are able to help with the building of the documentation I can let them have full editor rights and we can get going, all they need to do is to register on Codeplex.com and let me know what their username is. Codeplex.com is a Microsoft site and so you can even use your Microsoft live login if you have one.

Once we have some documentation I can update the program so that users accessing help within the program are taken to the correct page in the documentation in their browser. This then allows the documentation to be enhanced and tweaked independently of releasing a new version of the program.

 

[Tim]

I've joined. I'll PM the user name.

 

[Alexander Bisset]

I've created a start to the documentation, currently it's no more than a list of headings needing information added. Tim has started to add some information to the site so its starting out.

You can see progress here.

 

[Alexander Bisset]

Thanks for post #9, that clears up a few things. I notice on the Census Tab you also have the "View Lost Cousins census Report", which appears to give the same result as the button on the Lost Cousins Tab. I was just wondering, on the Census Tab, you can filter by surname, can that be also utilised for the report on the Census tab? i.e. a Lost Cousins census report on just one surname?

Yes that is the plan.

 

[Tim]

Sounds good.

 

[Tim]

I've created a start to the documentation, currently it's no more than a list of headings needing information added. Tim has started to add some information to the site so its starting out.

You can see progress here.

Guys, I've added some info to some of the start pages and also the Lost Cousins Tab, can you check them out and see how they read and any constructive thoughts please?