Managing reference checking

[Alexander Bisset]

I’m wondering how people remember they have checked a reference in Lost Cousins. Especially if you have hundreds of entries entered.

As Peter has mentioned many times it is vital that users go through each household and check that the reference is correct. Otherwise your Lost Cousins are never going to connect with you. However this can be a tedious process and I can’t see anyway to remind you which entries have been checked and which haven’t.

Since my family is mostly Scotland I typically have only one option to enter my references so I probably have less entries than most members. So I can only imagine that having a large number of entries becomes really tricky to manage and to ensure you have checked every entry.

I’m hoping there is a viable method of checking and that everyone checks. However I suspect similar to taking the time to enter every reference many members have yet to check more than a few households and might even have forgotten which ones they are or how far they got.

My fear is that the system breaks down if you don’t make sure you have the right entries and because I can’t see how to check I’ve confirmed an entry I suspect that like me most members don’t check.

I’d love to be able to automate this with FTAnalyzer however I can’t as there is nothing to say an entry has been checked or not on the website. So whether you are a pencil and paper researcher or a fully computerised researcher. I can’t see how anyone keeps track of who they have checked. To me it’s far far to manual a process to bother with. So I assume I’m missing some simple step.

Can anyone help out.

 

[AdrienneQ]

Alex after uploading my first set of data I put the entries in Lost Cousins in household order and checked all that were marked as having been uploaded via FTAnalyzer .
This will not work on my subsequent uploads so I will be using your export to Excel option from the Potential Records report to give me a list to work through on LC
I have data to check before I do this including the records on your Invalid Census reference report.

 

[Bryman]

Perhaps one simple way to lessen the problem and make it more manageable might be to introduce a user selected limit to the number of updates performed at any one time. Two alternatives come to mind . . .
1. A new field for the user to enter a number after the potential list has been examined. Then only the first x records would be attempted.
2. Introduce a user ability to delete, or mark to be ignored, any particular record listed for update.
The records to be processed then to be highlighted before the go command is activated.
It would then be up to the user to check their My Ancestors page, in Date of Update sequence, before submitting a further update request.

If the updates are not checked then the database will very quickly get corrupted. This very powerful new facility of FTA must be used responsibly.

 

[Helen7]

Previously I have checked each household as I manually entered them and corrected where needed. After successfully using FTAnalyzer to upload 66 entries to my LC account two days ago, I went through the marked households. However, I found no less than 21 of the new entries matched with a new cousin and I was unable to alter these, without deleting and re-entering them. This rather defeated the object of auto-uploading them in the first place, and I am reluctant to delete and re-enter as I might lose the matches, so I have left them as is for now.

In retrospect, I should probably have checked them all before clicking the 'check for new matches' button - clearly I was too impatient!

Of these 21 entries, 13 show minor discrepancies - usually birth year slightly different from census (though a couple of them were 5 years out) and variations in name spellings, though one had the wrong folio number - due to Ancestry mis-transcription - so is a clear error (on my part, not FTA obviously).

I wonder if the matched person has the same errors in their list? And if in correcting the errors in my list, they would no longer match with me? Perhaps Peter can advise on this. (I have pressed the contact button for them but yet to receive a response).

 

[Helen7]

Perhaps one simple way to lessen the problem and make it more manageable might be to introduce a user selected limit to the number of updates performed at any one time.

I fully agree with this. Uploading too many records at one time makes for a daunting checking task. Fortunately I only had 66 in my account and 76 in my husband's, but even that took a little while to check (mostly only 1-2 people per household).

One thing I did was to upload the blood relations and the 'by marriage' separately which splits it up a bit, but having a (user selectable) number limit or allowing the user to de-select particular records for upload is a very good idea.

 

[Alexander Bisset]

Perhaps one simple way to lessen the problem and make it more manageable might be to introduce a user selected limit to the number of updates performed at any one time. Two alternatives come to mind . . .
1. A new field for the user to enter a number after the potential list has been examined. Then only the first x records would be attempted.
2. Introduce a user ability to delete, or mark to be ignored, any particular record listed for update.
The records to be processed then to be highlighted before the go command is activated.
It would then be up to the user to check their My Ancestors page, in Date of Update sequence, before submitting a further update request.

If the updates are not checked then the database will very quickly get corrupted. This very powerful new facility of FTA must be used responsibly.

Having a user select records may be possible however it is even MORE manual processing than what we have at present and my aim is to REDUCE the amount of manual effort not increase it. Adding a maximum upload would work and would be relatively simple to add.

However it still begs the question how do you know you have actually checked a record? That was my query. To me there is no means of knowing you have checked the record or not thus no means of keeping track of whether you need to check the record.

 

[Alexander Bisset]

Consider the following. On the my ancestors page there is a button that links to the FMP record. What if you could manually tick a box on the website to say that record is checked then everyone in the household gets a green button that links to FMP record instead of a black one. Would that not make it really easy to see which ones were unchecked and which ones were checked. Then it wouldn't matter how many were uploaded. You'd see at a glance what needed checked.

Perhaps more importantly for the less enthusiastic users it would provide a simple means to remind them they have records needing checked. Having users check records and having that process be simple helps everyone and improves the overall quality of the data at the same time. All it takes is a tick box on the website.

 

[peter]

However, I found no less than 21 of the new entries matched with a new cousin and I was unable to alter these, without deleting and re-entering them. This rather defeated the object of auto-uploading them in the first place, and I am reluctant to delete and re-enter as I might lose the matches, so I have left them as is for now.

In retrospect, I should probably have checked them all before clicking the 'check for new matches' button - clearly I was too impatient!

In a case like this you ideally need to connect with the new cousin and agree with them what the correct version is - then both make the changes.

The important thing to remember is that there could be other cousins who neither of you are matching.

 

[Tim]

I agree with you Alexander, having a visual aid on the LC website would be extremely helpful to people who manually enter or use FTanalyzer to load their relatives.

It would also help you Peter, you would be able to see how many people have checked and have a greater understnading of the accuracy of your database.
You could even show a user what their accuracy was, on the My Summary page, you could say something like "You have only checked 42 out of 101 Households"

I understand space might be a premium on the My Ancestors page, but you could have a field on the Edit Ancestor page? And then as Alexander suggested, once the radio button is ticked there, it changes the grey arrow button on the My Ancestors page to Green?

 

[peter]

There is already a fairly simple solution - the My Ancestors page can be sorted in order of Date of entry so it's easy to see the most recent entries, which will be the ones that haven't been checked previously (if necessary the data and time of each entry can be found by clicking Show more detail).

For the majority who input their relatives manually (and, realistically, are likely to continue to do so) the best time to check their entry is at the rime of input, specifically after entering the head of household - since if there is an error in the census references only one entry needs to be changed.

Anything further just complicates things - and is likely to create additional work for those who have been conscientiously checking their entries all along.

From a practical point of view it would further increase the complexity of the My Ancestors page, and whilst it's hard to estimate what the impact on processing time would be, it could potentially lead to time-out errors, something we've only just managed to get rid of after two years of problems for our most dedicated users.

There are also programming issues but I won't go into them at this stage.

 

[Helen7]

For the majority who input their relatives manually (and, realistically, are likely to continue to do so) the best time to check their entry is at the time of input

I agree, this strategy has always worked for me.

 

[Alexander Bisset]

Fair enough I'll withdraw the verification tab and stop all further development on Lost Cousins features. There's simply no point in me wasting my time if no assistance is going to be given. I tried but I really don't know why I bothered.

 

[Tim]

Peter, how many manual entries were added to your database before you added the grey check arrow? There must a huge amount of people who have added thousands of households that have never been checked.

Your fairly simple solution does not work for your existing members, it only works for new members.

If people have 2 ways of adding data to LC

1. Manually add the head, 20 seconds, check the census refs, 20 seconds, add 5 remaining members 50 seconds which comes to 1 min 30 seconds, (and I think I've been overly conservative here)
2. Use FTAnalyzer to load 6 people, 2 seconds, check the census refs, 20 seconds, which comes to 22 seconds.

Why are they not going to choose or want to use the quicker option? How many of your members or potential new members have thousands of relatives they could add but are put off by the time it takes to manually add hundreds of households? 99% of people use family tree software and all the information they need is readily available, and has been demonstrated, loads very quickly and easily into LC.

As you correctly mention in your Newsletters, your members tend not to be newbies, they are established genealogists with huge trees. Maybe you should do a poll and ask your Newsletter readers why they don't load their relatives? Because if I came to your site today with over 3,600 people to load, then I'm pretty sure I wouldn't do it. Maybe I'd load my direct ancestors.

You have a great opportunity here to take a huge step forward in the number of records added, the number of active paying members and the accuracy of your records. I implore you to work out a simple solution that works for you and works for FTAnalyzer load.

 

[Bryman]

On the my ancestors page there is a button that links to the FMP record. What if you could manually tick a box on the website to say that record is checked then everyone in the household gets a green button that links to FMP record instead of a black one.

Nice try but just one small fly in the ointment. Not all census records on My Ancestors page have the FMP check arrow. What about them?

I would also like an improvement to say that red exclamation marks have been checked, without losing them if the record is not then matched.
I have about 30 such near matches which I suspect that the other members have not checked or for whom the mark has disappeared. These are almost certainly individuals which should be matched as other family members are. It doesn't stop cousins from making contact unless that is the only record in the household but does leave the page rather messy. However, this is not top priority and I am trying hard not to load up Peter with an increasing number of program changes.

 

[Bryman]

Fair enough I'll withdraw the verification tab and stop all further development on Lost Cousins features. There's simply no point in me wasting my time if no assistance is going to be given.

You are certainly not wasting your time providing this new feature in FTA. It is a great idea but we have to find a way of maintaining the integrity of the census data entered. I prefer to enter the census references as I find them and then use FTA to alert me to any that I have missed. Others may work differently and I appreciate that some members have a huge backlog to be processed and are unlikely to enter them without automated help. However, the current need for checking them after they have been mass entered would rather negate the benefit.

 

[jorghes]

As someone who uploaded over 1000 records when FTA added it's auto upload feature, I must say it is quite tedious to check everything. However, what did was as follows (and it will sound extra tedious) - I took screenshots of the records I had added with the little "added by FT Analyzer" marker, and as I have an iPad Pro, when I get a chance I open the photos and as I check I cross off the names (I can use my Apple Pencil or my finger to actually highlight the name on the image). If I have completed all the names on the image, I delete that image from my iPad, and thus I know I checked that one.

The issues I have come across so far:
- not all of a household were added sequentially, and occasionally I have had to search through the list for the rest of the household as it's easier to check details that way. Unfortunately I do not always then find those names that are out of order in my photo list (to cross them out), and occasionally I have caught myself double checking records.
(alternatively doing this I have found people that are on the record, but somehow have not made it onto my tree... which is curious; and found a bunch where for some reason the records have been deleted off their entry on my tree and I have had to reattach everything).
- A large portion of the records were from the 1940 US census, which doesn't have a little grey checking arrow, which I will then have to check manually - I am not looking forward to this part!!

I think I'm about half way through, but I still have about 30 images on my iPad... and that doesn't count any further records I have added using FTA after the point at which I took the photos - as I have kept checking the ability of the Mac beta to upload. (the majority of the images left are 1940 US census...)

 

[BrianM]

Alexander ... a thought ...
Can you, at the time of load, check the Census reference (equivalent of pushing the grey arrow - it sounds like you are able to, you just have the issue of determining whether it has been done previously?) and if it fails, delete the record and report the error/issue? Upshot is that references for loaded records are checked at the point of load, and invalid references are detected and removed before the load process is finalised. It retains the 'integrity' of the LC database, and does the 'recommended' checking that Peter wishes all people who add entries do. Downside is of course the additional processing time required during the FTA load (currently about 2 secs (?) per record, likely to blow out by a factor of ???).

 

[Alexander Bisset]

Nice try but just one small fly in the ointment. Not all census records on My Ancestors page have the FMP check arrow. What about them?

Trivial. The icon would not be green as the icon would not exist.

 

[Bryman]

The icon would not be green as the icon would not exist.

So how would one know that the update had been checked and verified?

 

[canadianbeth]

Bryman mentioned the check arrow being FMP. I use Ancestry records and have noticed on one or two occasions when checking that some information is different, generally with the spelling of names. Should I be then editing my entries to match the FMP one?