GAO: DHS cooks performance results for new radiation detectors

GAO: DHS cooks performance results for new radiation detectors Congressional auditors reiterated Tuesday that the U.S. Department of Homeland Security should postpone a decision on full-scale procurement of advanced radiation detection machines until further testing can determine whether the next-generation technology can be relied upon to prevent illegal cross-border transit of nuclear weapons and materials.
   The Government Accountability Office has charged DHS with using biased test results to promote the capability of Advanced Spectroscopic Portal (ASP) monitors that can distinguish between dangerous and normal sources of radiation in ocean containers and large trucks at ports of entry.
   The Domestic Nuclear Detection office 'conducted numerous preliminary runs of almost all of the materials, and combinations of materials, that were used in the formal tests and then allowed ASP contractors to collect test data and adjust their systems to identify these materials. It is highly unlikely that such favorable circumstances would present themselves under real world conditions,' the GAO said in prepared testimony for the House Energy and Commerce subcommittee on oversight and investigations.
   DNDO has ASP development and procurement contracts for 1,200 detection machines with Raytheon Co.-Integrated Defense Systems, Thermo Electron Corp., and Canberra Industries.
   The GAO also charged that tests were too easy because the DNDO did not use lead or other materials that terrorists would likely use to shield any radioactive material hidden in a conveyance. DNDO rejected recommendations from the Department of Energy and national laboratories to conduct more rigorous testing with masking agents because of concerns about meeting the original deadline for DHS Secretary Michael Chertoff to certify the machines by June 26.
   In August, the chairmen of the Senate and House Homeland Security committees called on DHS to postpone the $1.2 billion acquisition of large-scale nuclear radiation monitors for ports and other facilities until questions about their reliability, performance and effectiveness have been resolved. They asked DHS to involve the GAO in evaluating the program.
   But DNDO Director Vayl Oxford stood by the organization's 'very rigorous test campaign' at the hearing and said it will work to better explain the test approach and results.
   The GAO has criticized DHS in the past year for moving ahead with the acquisition even though testing has shown that the Advanced Spectroscopic Portals frequently fail to identify the type of radioactive isotopes detected, making them no better than current technology at sorting containers with products emitting natural radiation and those that might contain a radioactive weapon or fissile material. The watchdog agency also has previously criticized DNDO's cost-benefit analysis for relying on assumptions about performance levels rather than actual tests.
   Earlier this year, Congress requested the GAO audit the reliability of the radiation portal monitors in operational settings. DHS is conducting field tests with the next-generation machines at the New York Container Terminal; Port of Los Angeles-Long Beach; Laredo, Texas; and Detroit. The goal is to reduce the number of nuisance alarms and trucks that have to undergo secondary inspections.
   Oxford said he anticipates nuisance radiation alarms to drop from about 500 at the Port of Los Angeles-Long Beach to roughly 25 per day that inspectors would have to double-check with the new devices.
   Chertoff has said he will consult independent technical experts about the ASP test results before moving ahead with certifying that the systems are superior and worth procuring.
   Paul Schneider, undersecretary for management at DHS, told the subcommittee that field validations have been extended two months to obtain more data and that certification will not take place until the end of October.
   The GAO threw water on DHS's plan to engage outside scientists.
   'Because of concerns raised that DNDO did not sufficiently test the limitations of ASPs, DNDO is attempting to compensate for weakness in the original test plan by conducting additional studies — essentially computer simulations. While DNDO, Customs and Border Protection and DOE have now reached an agreement to wait and see whether the results of these studies will provide useful data regarding the ASPs' capabilities, in our view and those of other experts, computer simulations are not as good as actual testing with nuclear and masking agents,' the watchdog agency said.
   'We believe that DNDO's test methods — specifically, conducting dry runs and dress rehearsals with contractors prior to formal testing — enhanced the performance of the ASPs beyond what they are likely to achieve in actual use. Furthermore, the tests were not a rigorous evaluation of the ASP's capabilities, but rather a developmental demonstration of ASP performance under controlled conditions which did not test the limitations of the ASP systems,' the report said.
   The GAO also found fault with DHS for not objectively testing the performance of handheld radiation isotope detectors because it did not evaluate them according to how they are commonly used by border inspectors in the field. CBP normally sends readings to its Laboratories and Scientific Services group if readings in secondary screening are not definitive, but DNDO did not include this step in its test results. Initial deployments of ASPs are expected at secondary inspection locations at foreign and domestic ports to reduce the need for manpower to conduct handheld readings, provide greater inspection consistency and speed up cargo throughput.
   CBP and Energy officials complained about their lack of involvement in the planning and execution of the tests at DNDO's facility in Nevada, according to the GAO.
   Meanwhile, another group of federal investigators reported that cost overruns and mismanagement by the DOE have delayed completion of a facility sought by DHS to test sensors for detecting radiation at the nation's ports of entry.
   ASPs that can detect and differentiate radiation in large shipping containers and trucks are among the types of technology that DHS planned to evaluate and train personnel on at the facility.
   DHS wanted the DOE's National Nuclear Security Administration (NNSA) to finish building a nuclear and radiological countermeasures test site in Nevada on its behalf by February, but the project was halted two-thirds of the way through in August 2006 because contractor Bechtel had burned through $30 million of the $33 million of available funds.
   Although expedited design and construction methods increased project risk, the DOE's Inspector General said in a report last week that the NNSA failed to develop cost and schedule baselines for overseeing the project, allocate adequate contingency funds, monitor project status and communicate with DHS officials.
   Poor coordination of procurement and construction schedules, for example, caused a three-month delay due to the unavailability of equipment and building materials.
   Auditors also faulted the agency for unquestioningly accepting Bechtel's assurances that the project was on schedule and within budget. They determined that NNSA staff assigned to the project had little experience dealing with large construction projects.
   DHS has been criticized in the past for lacking the procurement staff and expertise to properly manage large contracts. The audit said DHS and DOE didn't coordinate oversight responsibilities until the project was two years underway. The lack of a clear agreement caused cost increases to be paid out of the contingency fund rather than DHS determining whether to increase funding or reduce the project's scope.
   DHS is now considering contracting the remaining work to the U.S. Army Corps of Engineers at a cost of $10.5 million, according to the report.
   'Even if an effective fix is implemented, completion of the project will have been significantly delayed and the cost will have substantially exceeded original estimates. More importantly, the delay may impact the nation’s testing capability to detect nuclear and radioactive materials in a variety of circumstances,' the inspector general said.
   The NNSA reduced Bechtel's award fee by $687,000 for poor performance, but should also have recovered some of the company's fixed payment of $1.2 million, the inspector general said.
   Bechtel was replaced on the project in July 2006 by National Security Technologies LLC.