My Two Census

Formerly the non-partisan watchdog of the 2010 US Census, and currently an opinion blog that covers all things political, media, foreign policy, globalization, and culture…but sometimes returning to its census/demographics roots.

Posts Tagged ‘quality assurance’

Transcript from Census Bureau Director’s latest press briefing…

Friday, July 9th, 2010

Here’s the transcript and info from the latest press briefing. Here are some quotes of interest from Dr. Robert M. Groves:

1. On the second risk—the software systems, the new management team—I can say
honestly now that, although we had a very shaky start with these software systems, with
management interventions that were wisely done from this team, with enormous
dedication from a bunch of software engineers, we have successfully processed
47,000,000 forms through this software system that was designed to do that. It worked. It
wasn’t pretty, but it worked, and we have successfully completed that phase. We have a
few tail-end things that we’re finishing up.

2. The first operation is the biggest we’ll operate, and that’s called Coverage Follow-Up.
That actually began in mid-April, and it should finish up by August 13th. We’re calling on
about 7.5 million households. This is exploiting an innovation in the 2010 Census. If you
remember your form, there were two questions. One, is there someone at your house right
now who normally doesn’t live there? And then for every person you reported, we asked
the question, does this person sometimes live elsewhere? For the houses that checked one
of those boxes, we’re going to call back and make sure, make double sure, that we’ve
counted people once and only once. It is these complicated households where people are
coming and going and living there sometimes and not other times that pose real
challenges to get accurate counts, so we’re calling back on those.

3. The second operation, The Vacant/Delete Check, is about the same size, about 8,000,000
households, and what we’re doing there is going back on a set of households that we’ve
visited over the past few weeks where, when an enumerator went up and knocked on the
door, they determined, he or she determined, that that house was vacant on April 1. We
want to make sure that’s right. We’re going to double check that. We’re going to go back
to that house and redetermine [sic] whether that’s a correct designation for the house.
And then there are other houses on our list that went out over the past few weeks, and
when they went out to locate the house, they saw an empty lot, the house had been
destroyed, or they couldn’t find the house. It looked like our list was inappropriate. And
they marked that as a Delete. We’re going to go back out to those and make sure we got
that right before we finalize the operation.

4. The third operation is called Field Verification. It will began August 6th and it will go
through early September. This is really our last operation in terms of time. It’s pretty
small. We’re going to about 400,000 addresses. This is a check on a set of cases that is,
itself, the result of our efforts to count everyone. So, in March and April, if you didn’t get
a form, we said you could go to a local facility and pick up what we call a Be Counted
form. People did that. Not too many people, but people did it. And on that form we asked
you to write your address. We’ve examined every one of those forms already. And
sometimes, when we look at the address, we can’t match it to anything we have on our
list. On those kinds of cases, we’re going to go back out. We’re going out to that house
and we’re going to make sure we can find it. We can understand the address, we know
what block it’s in, and we can place it correctly in that block.

5. One another note that is useful to make. If you’re out there, or if your audience has the
following thoughts, “Gee, I don’t believe I got a mail questionnaire. I know I didn’t send
it back. I haven’t had anyone knock on my door. I’m afraid I’m not counted.” We still
have a facility for you, an 800 number. 866-872-6868. If you press the right buttons, I’m
told, rather slowly, you will get connected to an interviewer who will take your data right
on the phone. And that’s still open. That’ll be open until the end of July, roughly.

6. This was a short form only census. In 2000, the short-form had a response rate about ten
percentage points than the then long-form. We were counting on this. This was part of the
success. This is really the only way we achieved that 72% mail out response rate, I’m
pretty sure. Secondly, remember we had a bilingual form that was sent to areas that were
disproportionately Spanish-only speakers. We’ve analyzed those data. That thing worked
the way we wanted it to work. It increased the return rate in high prevalence Spanish-
speaking areas, we’re pretty sure. It’s a complicated analysis that will take longer to do,
but we’re pretty sure that thing worked the way we wanted it to.

7. We have a lot of junk on the list.” We deleted about 4.1 million cases in 2010. In 2000, we deleted
6,000,000. We like that contrast. Right? It looks the list is cleaner on the Delete side.

8. In the 2000 cycle, we were able to do reinterviews [sic] on 75% of the interviewers. 75% of the
enumerators got at least one case in their workload redone and checked. We’re essentially
at 100% now; we’re 99. something. That’s a good thing. That means we can say honestly
that a piece of every Census worker’s work was redone, independently, and checked to
see if we found any departures from training guidelines. We like that result.

9. This Vacant/Delete Check will really nail that number, but right now we stand, as of today, we found about 14.3 million vacant
homes versus 9.9 in the 2000 cycle.

10. We have about 47,000,000 households, we have about 565,000
interviewers, it looks like the number of cases that we judged as so severely mismatched
that it could’ve been a fabrication incident is less than a thousand out of those 565,000.
This is, by the way, below what we expected. And we feel good about that, because we
know we’ve sampled work from every interviewer, essentially.

11. Now, the second question is about prosecution. We are not in the prosecution game, as
you know. When there are severe, endemic, large amounts of fabrication, then that’s a
matter where we would call the Inspector General, if they weren’t aware of it already.
They do an independent investigation, and then they would make a recommendation to
the relevant U.S. attorney to prosecute or not prosecute.

12. It is feasible, as the caller noted, that we would count someone both at a soup kitchen one
day and then we would visit an encampment, or a group of people sleeping under an
overpass. When we visit them in the evening, it is very common that those people are
worried about their own safety. They protect themselves in various ways, to make sure
they’re not harmed physically. It is common that when we visit those outdoor locations,
that we can’t get the names and age and race of each individual. They say essentially,
“We don’t want to talk to you.” As a last resort, in those cases, we enumerate, we count,
Person 1, Person 2–  that’s about the best we can do.

13. ANDREA ISHAL: I’m Andrea Ishal,  …(inaudible) Reporters. I wanted to follow up on
one question that came before, and then ask one of my own. You had said that there were
1,000 cases–  was that 1,000 individuals out of 585,000, or was it a thousand cases out of
the 47,000,000.

GROVES: What I wanted to say—and we’re still doing this, so I don’t know the final
numbers—but we’re confident that it will be less than 1,000 people who, in
reinterviewing cases they did, we have judged falsified those cases. That’s 1,000 out of
565,000 roughly.

14. The number, just for talking purposes, in talking about the marginal cost
of calling on a case and doing an interview is about a $57 a household or about $25 a
person. And those are numbers that we’re still working with. We’ll refine those numbers
based on our experience as soon as we collect all the data.

15. JEFF COONER:  The second question was, you were talking about being under budget,
so I wanted to know what the budget was and what we actually spent.

GROVES:  Yea, yea. Well, again, we’re not sure on this. But we’re coming in at the
Non-Response Follow-Up stage at about 70%-75%  of the budget. We’re not through
with that yet, so we’re not able to report on that. But that’s a significant cost savings,
we’re sure. The why of those cost savings are important to note, too. Part of it is our
workload was lower than we were prepared to do. These are good things. We had less
cases than we were ready to call on; that we thought we’d have to call on. The second
thing that happened was, we’re now pretty sure, that the work of this labor force that we
engaged was just smoother. We got cases in faster than we thought. We think the
productivity was greater. I’ve noted several times that we are blessed.

16. SARA HASAID: Hi, Sara Hasaid from AFP. You mentioned that the number of cases in
which you had to appeal to a landlord or a building manager to get information was
higher this time around. Can you give me any sense of actually what those figures are and
why that might be?

GROVES: I can. And it’s roughly 21% or 22% of the 47,000,000 that we went on to
knock on the door. And if you look, it’s a bit of apples and oranges. But if you look at the
2000 rate, that was about 17%, so it’s a little higher. Did you have a second question?

[off mic]

There are a lot of different reasons. This tracks trends and surveys. For those of you who
know a little about surveys, you know it’s harder to get a hold of us than it used to be.
People are at home less frequently, for a lot of complicated reasons. These 47,000,000
households, by the way, are the households that chose not to return the mail
questionnaire. These are really busy people. And so that’s part of it. And there’s a
reluctance in that contrast between 17% and 22% that we don’t know the components of
yet. People who open the door, they’re at home, but they say, “I don’t want to do this.”
And we go back repeatedly, we send different enumerators, and as a last resort, then we’ll
ask a building manager or a neighbor.

The big news from today’s press conference…

Wednesday, July 7th, 2010

I don’t have a complete transcript of the press conference available, but of the 565,000 enumerators involved in non-response follow-up (NRFU) operations, the Census Bureau estimates that 1,000 of these people fabricated data. In the coming days MyTwoCensus.com will demonstrate why this “1,000″ figure is too low and how additional data that has been compromised will be overlooked.

Census Bureau Press Release About Today’s Press Conference

Wednesday, July 7th, 2010

Here it is, more details and analysis coming soon:

Census Bureau Director Reports 2010 Censuson Schedule, Under Budget
Census Moves into Quality Assurance Operations to Help Ensure Complete,
Accurate Count

As census takers across the nation finish the 2010 Census door-to-door
follow-up operation, the U.S. Census Bureau has entered the quality
assurance phase, where select households around the country will be
contacted by a census worker. Three major operations occur this summer that
mark the peak of efforts to ensure data accuracy.

The 2010 Census is on schedule and significantly under budget but not
fully completed. The Census Bureau systematically re-interviews 5 percent
of the households that each census worker visits to confirm that all of the
600,000 census takers followed training protocols and produced accurate
data.

“We thank the American public for their participation in our
door-to-door follow-up phase and for their continued patience as we enter
the next vital stage of the 2010 Census,” said Census Bureau Director
Robert M. Groves. “We ask that if you are one of the few homes
re-interviewed, called or visited this summer during our quality assurance
operations, please take a few minutes to help us ensure that the 2010
Census is complete and accurate.”

Groves expressed confidence that census takers are doing a quality job,
and he reaffirmed that the current process enables the Census Bureau to
catch any errors or corner-cutting and initiate immediate corrections.

In the coverage follow-up operation, the Census Bureau calls households
to eliminate confusion about the number of people reported in a household
to make certain there are no missing or double-counted individuals. As the
nation experiences one of the highest vacancy rates in recent years, the
vacant/delete check operation requires census workers to visit households
that were listed as vacant on April 1 (Census Day) to double-check that no
individual has been left out. The field verification operation verifies the
location of addresses provided by Be Counted forms or through telephone
interviews to ensure everyone is counted in the correct location.

“Decades of census taking have taught us the importance of the quality
assurance phase, and we know crucial federal funding and congressional
apportionment relies heavily on our ability to produce an accurate census
count,” Groves said. “That is why these quality assurance operations —
inspired by our mantra to count everyone once, only once, and in the right
place — are critical to our country’s future.”

Census worker arrested for trespassing…how will the Census Bureau respond?

Friday, July 2nd, 2010

Most likely, the Census Bureau will come to the aid of its employee, but it will be interesting to see how this incident plays out in court. In a recent incident in Hawaii, the federal government staunchly defended their field worker. Let’s see how the following story plays out:

Count Me Out
Rosendale Census Worker Receives Summons for Trespassing
By Rochelle Riservato and Tod Westlake

ROSENDALE – A United States 2010 Census Bureau Quality Assurance fieldworker was involved in an altercation at a residence in Tillson on June 1, which then resulted in the bureau fieldworker being served with a “criminal summons for trespass.” The fieldworker was then issued an appearance date of Wednesday, July 21, in Rosendale Town Justice Court.

Apparently, the fieldworker — who would speak to the Journal only on the condition that her name be withheld from publication — was performing a re-interview visit at a Rosendale home, and was requested by the homeowner to leave the property. The census fieldworker stated that she got back in her vehicle to finish filling out “refusal paperwork” and left the premises.

“I want these charges expunged from my record,” said the fieldworker, “I was only doing my job — a ‘no-response’ is the only reason someone like me has to go at all.”

But a statement by the homeowner in question tells a different story. The homeowner says that she told the fieldworker that she had already filled out the form, but was told by the fieldworker that this was a “follow-up” visit. When the homeowner refused to cooperate, asking the fieldworker to leave the premises, the fieldworker responded that she was a “federal officer” and that she would “sleep in her car in [the homeowner's] driveway” until the homeowner complied. The homeowner repeated her request that the fieldworker leave her property, and the latter “laughed at her,” according to the statement. The incident took a total of 20-25 minutes, the statement says. The homeowner says that the fieldworker went so far as to suggest that the homeowner would be “wearing cuffs by the time this was over.”

The fieldworker says that, two weeks after the altercation, she came home to find a Rosendale police vehicle blocking her driveway when she arrived home from work.

“I said ‘Hi Officer, what’s going on?’ I had no idea he was coming to arrest me,” she says. “The officer told me I was under arrest for criminal trespassing and handed me a summons signed by Justice Robert Vosper.”

Chief Perry Soule of the Rosendale Police, however, said this was not an arrest. The officer was simply delivering a criminal summons from the justice court. Soule said that his office typically does not see complaints against census workers, and that these are referred to the justice court.

Click here for the full article.

Investigative Series: Spotlight on Harris Corp. (Part 3)

Thursday, May 21st, 2009

Here is a first-person account (written by a highly qualified Census Bureau employee who has requested anonymity) submitted to us about the Harris Corp’s handheld computers that have been used in the field by the Census Bureau’s address canvassers during the first stage of 2010 Census operations:

I’d say the biggest bug in the handhelds was due to the government trying to assure privacy. They had the handhelds set up to hide information from us after we entered it. So after declaring a street or an area “done,” the computer hid that information so we couldn’t go back to check, or to compare or verify our work. (So, we learned to avoid marking things “done” until we were absolutely sure we wouldn’t need to check back.)

The handhelds provided an advantage, in that they served to level the information-recording playing field amongst the canvassers. When the GPS was working (which was 99% of the time for me, the only time I had a bad signal was in a very wooded area), it made it quite easy to “map-spot” all the residences, and those spots will be used by USPS workers when delivering the census. Remember that in addition to the easy, obvious residences, there are plenty of residences that aren’t so evident. Cabins at the end of dirt roads, where people live there but have no mailboxes in favor of a PO Box. Trailers parked in driveways with separate families renting space. Rental apartments in the back-rooms of businesses or the basements of libraries. Our handhelds let us map all those places, and made it possible for everyone’s map-spotting to be equivalent. If we had been doing pencil-and-paper mapping, each canvasser’s information would have been different, because each would just be guessing where on the map each house was. Since we all had the same GPS technology, we’re guaranteed to be mapping residences at the same quality level. Also, by using the GPS/handheld technology, we probably cut our work time down by 3/4.

So I definitely think adding the handhelds was a good idea, it’s just too bad they implemented it poorly via a custom-contractor.