Last week, Census Bureau Director Robert M. Groves said to Fox News that you can “trust 2010 Census data.” What our director fails to tell us is that the two software applications have operational problems that will ultimately lead to inaccurate data. Just spend a day working in PBOCS, the Paper-Based Operational Control System which processes enumerator questionnaires from the field, or MARCS, the Matching Address Review Coding System which shows a data capture of every questionnaire that was scanned at the Baltimore Data Capture Center and you will see the poor quality of work. Thousands upon thousands of questionnaires are being scanned that show conflicting or incomplete data such as: vacant housing units with a population count, incorrect enumerator IDs, occupied housing units with no demographic information and the list goes on.
During the peak of the non-response follow-up (NRFU) phase of 2010 Census operations (around mid May), the Census switched to a shipping application built off a PeopleSoft/Oracle interface in order to take the load off PBOCS. Although this was a good thought in theory, the application allowed questionnaires to be shipped that were not even checked in PBOCS. In the final closeout days of the operation, PBOCS claimed many questionnaires were not checked in even though enumerators fervently claimed they turned them in. Fortunately some of those were found in MARCS having been received at the data capture center but never scanned for shipping nor checked in. However because there was such a bottleneck sometimes a few weeks between the time they were shipped and scanned; some questionnaires that never showed in MARCS were re-enumerated. Sometimes PBOCS would just revert some cases back to not being checked in. In a mad dash to finish and meet deadlines enumerators submitted second versions of questionnaires with little or less than accurate data replacing what may or may not have been originally submitted. Immediately after offices finished NRFU, headquarters closed the PBOCS to the local census offices to prevent further glitches.
As it has been mentioned time and time again, the Census never made it clear what constituted a completed questionnaire. In such a recession, employees were promised more work if they finished quickly so experienced and resourceful field staff took advantage of the three visit rule sometimes making visits in consecutive days or all in a one day before going to a proxy. Local census offices managers, RCC supervisors and managers developed their own rules which were verbally communicated to field staff. These included guesstimating the population count and allowing enumerators to submit Enumerator Questionnaires (EQs) with little or no demographic information. Since performance was purely based on how many questionnaires get checked in; those who submitted hundreds of forms with nothing on the inside of the questionnaire were rewarded with more work.
On the quality assurance end, the staff attempted to examine the data collected for falsification and poor work quality. However reinterview only has been able to find those who intentionally falsified data. An enumerator can submit inaccurate or incomplete data and practically get away with it.
Most enumerators will be tempted to submit inaccurate data when they cannot gain access to the building, speak to a household member or knowledgeable proxy after repeated visits. The reinterview telephone clerks and field staff have to prove definitively by gaining access to the building or speaking to a respondent who said the interview was never conducted. But in reality the reinterview staff can never access the building, or with large apartment buildings sometimes a proxy is asked about hundreds of units and may not remember if the original interview occurred. Most of these bad data cases have little or no information or wrong information: no names, ages, Hispanic origin, race and sometimes not even a person count. But quality assurance staff have either been told to mark them refusals with an unknown population and check them in.
In the rare instance that the Census Bureaus’s quality assurance (QA) operations do suspect data falsification or inaccuracy, finding the culprit is difficult. There are thousands of questionnaires where the enumerator ID numbers are being read incorrectly at data capture. This invites data falsification in two ways. If a questionnaire is found to be inaccurate or falsified then it is impossible to find the culprit. If quality assurance staff does find an enumerator is submitting falsified or inaccurate work, they can not examine the other questionnaires the enumerator completed because many questionnaires do not have a valid enumerator associated with it.
In the current Vacant/Delete check phase of 2010 Census operations, while the agency covered up their own software problems by closing access to PBOCS, they have also created problems. For hundreds of questionnaires where enumerators clearly marked them vacant or deletes without visiting them LCOs cannot access the system to research who actually submitted this erroneous work.
Most of this is happening now in your local census offices across the country as the re-interview phase winds down. This is because of a huge backlog of EQs that were sent into re-interview, hundreds of outliers, and the slowness of MARCS. This inaccurate data is another smear of shame for the Census Bureau. For Dr. Groves to say that we can trust 2010 Census data is merely a cover-up.
Here are some e-mails sent to 2010 Census managers across the nation that detail the aforementioned problems:
07/18/2010
ATTENTION : 2010 Census Managers
SUBJECT: 1- PW Flags randomly appearing or disappearing on the Select Enumerator screen
2- Loss of notes in the LCO Notes panel on the Evaluate Case screen
3- Cases with missing person data from the 400,000 pushed cases
ACTION: Please share the information with the appropriate field staff
1. PW Flags randomly appearing or disappearing on the Select Enumerator screen
As a result of a MaRCS fix, the PW flag may have been working erratically. It has been reported that the PW flag on the Select Enumerator screen may have disappeared from the screen for already worked enumerators or may have appeared in cases for an enumerator the MaRCS clerk had never worked. This was a temporary issue and has been corrected. For those cases that this issue may have happened, please inform the AMQA they would need to remove the PW flag for the cases where the enumerator has not been worked in MaRCS OR asking the QA Clerk to click on the Edit pencil icon for the enumerator they have been working to reactivate the PW flag if it has disappeared.
2. Loss of notes in the LCO Notes panel in the Evaluate Case screen
As a result of the MaRCS performance issues that LCOs are experiencing, some screens are loading slowly. To avoid losing the notes entered in the Evaluate Case screen, the MaRCS clerk needs to wait until the page has fully loaded. A page is fully loaded when the “Please wait for page to respond” message disappears in MaRCS or when the Windows browser loading indicator (it shows as a progressive number of green squares) at the bottom of the browser also disappears. Please also remind the LCOs to enter the notes in the LCO Notes panel before assigning a final outcome on the case and to save these notes often so they are not lost if the MaRCS session times out.
3. Cases with missing person data from the 400,000 pushed cases
NPC noted that a portion of the 400,000 cases pushed for processing have blank person data in the original interview or the reinterview in cases where the unit status (US field in Review Data screen) shows occupied (OCC). Most if not all of these cases will be deferred to the LCOs due to different unit statuses between the original interview and reinterview. An example of this situation might be, the original interview has an unit status of occupied with 3 people living at the housing unit and the roster and demographic information is blank; and the reinterview shows that the housing unit is vacant (thus no roster or demographic information shown).
The MaRCS clerks should investigate these cases as any other case in LCO Review. For these cases, the MaRCS clerks should focus their investigation on the unit status of the housing unit, determining which one might be correct. When the MaRCS clerk determines the correct unit status, then they should turn their investigation on what might have caused the discrepancies in the data and assign an outcome code based on the investigation results.
07/15/2010
ATTENTION : 2010 Census Managers
SUBJECT: MaRCS NRFU users account maintenance
ACTION: Delete unused MaRCS accounts by noon, Friday 7/16/2010
MaRCS is experiencing performance issues due to the exceeding the number of users accessing and using the system at the same time. Per our teleconference today, attached below are the tallies by LCO of MaRCS accounts issued to users in the LCO. Please review the number of users in each of the region’s LCOs and delete the accounts that are no longer needed.
IMPORTANT – MaRCS accounts should be used for coding MaRCS cases. Limit or eliminate MaRCS uses for purposes other than coding MaRCS cases. Staff assigned to work MaRCS cases are the only staff allowed to have a MaRCS accounts in the LCOs.
The AMT can delete the unused accounts in the LCO. The RMQA needs to work with the AMQA to identify and delete the MaRCS accounts that are no longer needed. For example, we have noticed multiple AMQA roles for a single LCO. It is preferable to only have 1 AMQA role per LCO, as this is the person that has the responsibility to Hard Fail a case. LCOs may have, in rare cases, more than 1 AMQA role if the AMQA has a backup or if there are other AMQAs working shifts.
The AMT instructions to delete users in MaRCS are in their AMT Manual D-650.1, lesson 6. The RMQA can also ask the LSC to run the D-1311M User Role Report to verify user roles and that unused accounts are deleted.
After the accounts are deleted, the MaRCS contractor will measure system performance and inform us if this resolved the issue. Until further notice, please inform the LCOs to use, at most, 4 accounts per LCO OR use accounts not to go over the number of LCOs times 4 per region, the allowed number of MaRCS users.
07/09/2010
ATTENTION : 2010 Census Managers
SUBJECT: Start of the processing of 400,000 cases in MaRCS with data capture issues
ACTION: Please share the information with the appropriate field staff
As mentioned in the last RMQA teleconference, MaRCS held from processing about 400,000 cases that had a data capture problem. The data capture problem was in the population count where a scanning error, as an example, might have returned a population count of 74 when the actual count is 4. These cases were not processed because MaRCS was waiting for a continuation form where one was likely not needed.
MaRCS will start processing these forms starting on Monday, July 09, 2010 and should be finishing by the end of the week. These forms will likely be deferred to NPC from computer matching because the population counts will not match. It is expected that NPC will resolve the majority of these cases because as long as the roster and demographic information matches, the NPC clerks will pass the case.
It is not expected that the LCOs will get to code many of these cases. However, if they do get some of these cases, please remind the LCOs to ignore the population counts and, if the roster and demographic information matches, then pass the case. If the roster and demographic information does not match, then the MaRCS clerk needs to conduct an investigation on the case as any other case in LCO Review.
The other issue this should resolve are the cases that may be showing in the D-3421M Completion and Data Capture Report as not being data captured when there is information in PBOCS that the case was worked and shipped. It is expected that as these cases are processed, many cases showing in this report will be removed.
If you have any questions please contact Hector Merced or Vance Davis at 301-763-8822 or email fld.quality.assurance.branch@census.gov
07/02/2010
ATTENTION : 2010 Census Managers
SUBJECT:
1. Hard Fail Recommendation screen reminders
2. Applicant ID capture error – new known issue and workaround
3. Handling cases where the Address panel information in the Review Case Data screen is outside the LCO or RCC boundaries
4. Reminder on handling duplicate D-1282Ms
5. Update on cases not showing in PBOCS when a D-1282M exists in MaRCS
6. MaRCS clerk observation forms for both UE and NRFU
ACTION: Please share the information with the appropriate field staff
1. Hard Fail Recommendation screen reminders
Some regions have informed us that Hard Fail cases are not showing in the D-831M Hard Fail Report after the AMQA assigns a hard fail code to a case. This is due to the AMQA not entering notes in a timely manner in this screen (MaRCS times out) or exiting the screen before clicking the Save button. Please remind the AMQAs to be prepared to enter the notes and the LCO managers’ decisions prior to coming to this screen. It is suggested the AMQA has the notes ready in a notepad so they can quickly be entered on the screen along with the AMFO/LCOM decisions. The notes for a hard failed enumerator should not be lengthy since all LCO managers are in agreement with the outcome.
Not entering and properly saving these notes in this screen has also affected the D-831M Hard Fail Report. This is a defect that the MaRCS contractor is fixing today. An updated report with these cases should be available early next week. Also, as a result of this defect, D-1282M Transcription Reports were not generated for these hard failed enumerators. The fix to the report will also correct this defect, so LCOs should expect next week D-1282Ms with the completed eligible cases for the hard failed enumerator that needs to be reinterviewed.
2. Applicant ID capture error – new known issue and workaround
There is another known issue where valid applicant IDs and names show in MaRCS cases but the enumerator showing in the case does not work in that LCO. The rest of the data displayed for the case will belong to the LCO and the only inaccurate data is the applicant ID and name of the enumerator in the case. This happens when the applicant ID was incorrectly captured at the data capture center and it happened to match a valid ID from another enumerator in another LCO. The MaRCS clerk needs to review this case as any other and assign a final outcome code based on the case investigation (PASS, SOFT FAIL, DK/NO SUSP, or DK/SUSP).
If the MaRCS clerk reviewing the case is recommending to hard fail the case and the LCO managers agree to hard fail the case, please DO NOT HARD FAIL THIS CASE . Doing this will cause the enumerator outside the LCO being flagged as a Hard Fail enumerator. Have the MaRCS clerk Soft Fail the case. Using the case ID, please look if the LCO can identify the enumerator that actually worked the case in the LCO (or the RMQA can send the case ID to QAB to get that information). Once the correct enumerator is identified for the reviewed case, the AMQA can then Non-RI Fail the enumerator. This will ensure the right enumerator is hard failed and the completed eligible cases for this enumerator are reinterviewed.
No action is required if the reinterviewer name and applicant ID displayed in MaRCS is outside the LCO boundaries. The Reinterview panel information in the Review Case Data screen will belong to the LCO.
3. Handling cases where the Address panel information in the Review Case Data screen is outside the LCO or RCC boundaries
Some regions have said that they have cases from other LCOs or are outside the RCC boundaries. This is a known issue that happens for added housing units during NRFU. This is another data capture issue where the LCO was incorrectly captured for the added housing unit. There is no viable solution to transfer these cases to the appropriate LCO. Please instruct the LCOs to PASS these cases and include in the Notes the reason for the pass is the case is outside the LCO/RCC boundaries.
4. Reminder on handling duplicate D-1282Ms
This is a reminder to the LCOs to ignore the D-1282Ms that are duplicates. There might instances where MaRCS created 2 or more D-1282Ms for the same case ID. Please inform the LCOs to reinterview only one of the cases and to ignore all other possible duplicated D-1282Ms.
5. Update on cases not showing in PBOCS when a D-1282M exists in MaRCS
We got confirmation that MaRCS has passed all information to PBOCS as of 6/29/2010. From now on, the sponsor division will monitor that PBOCS receives the data from MaRCS and will inform QAB when PBOCS did not acknowledge receiving the data. We will inform the regions when the MaRCS cases were not received in PBOCS and provide guidance when this happens.
Also, DOTS staff will send back to the LCOs the Remedy tickets created when the case exists in MaRCS and not in PBOCS. The LCOs will be asked to see if the information is in PBOCS, as we have been given confirmation the information from MaRCS was acknowledge in PBOCS as of 6/20/10.
Unless QAB sends information to the regions that PBOCS did not acknowledge the data, a case not appearing in PBOCS is a PBOCS issue and not a MaRCS issue. Please inform the LCOs to submit the Remedy tickets to PBOCS and not MaRCS.
6. MaRCS clerk observation forms for both UE and NRFU
We have been told that MaRCS observation forms have been sent to NPC along with the NRFU enumerator observation forms. Please ask the LCOs not to send to NPC the MaRCS Observation forms. QAB will soon issue a disposition ops log for these forms and all other forms used in the investigations.
If you have any questions please contact Hector Merced or Vance Davis at 301-763-8822 or email fld.quality.assurance.branch@census.gov
07/01/2010 – New ops log for July
ATTENTION : 2010 Census Managers
SUBJECT: Clarification on 6/30/2010 ops log (Selecting additional cases for supplemental reinterview — Urgent Request)
ACTION: Please share the information with the appropriate field staff
Many of the regions have said that some of the cases for this special project cannot be sent to supplemental RI. The RMQAs need to check that the LCOs followed the following steps before sending the case IDs to the QAB branch as invalid case IDs. There are 4 possible reasons these cases cannot be sent to reinterview–the case has an invalid applicant ID, the case does not exist in MaRCS, the case has already been reinterviewed, or the case is ineligible for reinterview. All these scenarios are explained below.
The first step they need to do is check the case exists in MaRCS. This is done by clicking on the Case Search option at the top of the Welcome screen. The person selecting the supplemental case can then check if the case exists by entering the case ID in the Case ID box and ensuring the All Cases radio button is selected. If the case exists, please check that the Enumerator Name column has an enumerator name in it. If it does not, this is a case that has an invalid applicant ID and cannot be sent to RI. Please send these case IDs to the QAB branch. If the case search does not bring a case (the screen is blank for that case), then the case does not exist in MaRCS. Please send these case IDs to the QAB branch.
Also check in this screen if the case has already been sent to RI. The screen will show in the Outcome column the final outcome code assigned to the case. For this case, the RMQA needs to update the spreadsheet to record the results of this case. Please also send these case IDs to the QAB branch.
If the case exists, then the clerk selecting the supplemental cases need to be back at the Welcome screen to start the process of selecting the supplemental cases. At the Welcome screen, they need to click on Select RI Cases at the top (in the Menu bar). This will bring up the Select Supplemental RI Cases screen. The next step is to select the enumerator for the selected case. This is done by clicking on the drop down box labeled Select an Enumerator. It is likely that the first several entries on this drop down box are those cases with invalid IDs. Please ensure the clerk selecting the cases scrolls down the list until the enumerator name is found. When the enumerator name is found on the drop down box, click on it to bring up the cases for that enumerator. The clerk needs to scroll down the list until he/she finds the case. MaRCS will show a certain number of cases per screen, please ensure the clerks goes through all the screens with cases. This is done by clicking on the pagination links at the top right corner of the screen. Once the case/s are found, click on the Select column check box to send the case/s to supplemental RI. Please remind the LCOs not so select a precipitating case in the Enter Case Selection Details screen. The note the clerk can enter there can be “Special project.”
If after the clerk goes through all the screens looking for the case ID and the case is not included for the enumerator, the case then is ineligible for RI. Please send these case IDs to the QAB branch.
We do not know at this point if these cases will be replaced with other cases. We will let the regions know if we get replacement cases for these invalid case IDs.
If you have any questions please contact Hector Merced or Vance Davis at 301-763-8822 or email fld.quality.assurance.branch@census.gov
