BAD SCIENCE 40

SUMMARY: Hype is everywhere. You don’t need a degree in chemistry to examine a technical statement for truth, but it helps. Bad science is misapplying or twisting a good rule to make a point seem reasonable. We are degreed and experienced analytical chemists, and we see a lot of bad science published in sales literature and built into specifications. This is my soap box section to help you avoid the snake oil sales forces that are out there.

DETECTION LIMITS, ACCURACY AND PRECISION
One lab devotes a great deal of space defining these. Confidence Intervals, uncertainty amplification, generous assumptions and a mix of clever calculations are all applied to make one think that their air analysis program is very scientific, accurate, and precise. For example, they go on to say that they can measure 0.024 mg/m3 of oil mist + particulate. Really ? That claim deserves some scrutiny.

Since their target sample volume is 1 cubic meter, they are saying that they can accurately measure 24 micrograms of debris! To understand how small that is, if all that debris was oil, and I put it on a filter it would be 0.04 microliters; you would not be able to see it. Yes, micro-balances have the design capability to measure 24 micrograms, but that does not take into consideration the design and use of gas sampling hardware, and the handling of the filter from preparation to report. We could discuss the operational aspects of micro-balances, humidity control and filter characteristics, but that would be a waste of your time. Simply put, claims like this are obviously calculations rather than achievements.

QUALITY ASSURANCE IN THE REAL WORLD
To get accredited by AIHA or A2LA we all had to prepare QA Manuals. We are expected to follow these manuals daily, and to maintain proof of that. Are those expectations realized? I know of one medium sized laboratory that didn’t know how to write a QA Manual. So, they simply borrowed a one inch thick QA manual from another company, copied it, and put their name on it. It didn’t apply to what they did, and they never actually used it, but it got them accredited.

Here is another case history. We purchased an air sampling kit from a large well-known lab that does air sampling and analysis. We prepared a contaminated sample and sent it in to them. They reported back the correct Volatile Hydrocarbon content, however, they reported the Oil Mist + Particulate result as “clean, < 0.2 mg/m3”. The truth was that we had put enough oil on the filter to equal 8.0 mg/m3 ; they should have at least failed the sample. Apparently they never weigh the filters unless they look dirty. Isn’t that interesting (and scary). No, I am not allowed to tell you who they are. All labs have trade secrets, but not all labs take shortcuts.

PARTICULATE MAXIMUMS – more bad science
I have 3 Oxygen Compatible Air specifications (for Nitrox preparation) in front of me. One says 1 micron max, another says 2 microns, and the third has no size specification. But, the specifications do not say how many 1 or 2 micron particles it takes to fail the sample. (Even the FDA says how many insect parts you can have in a jar of catsup.) Nor does it discriminate between a particle of desiccant, sand or a piece of Teflon.

Can the examination be routinely performed and the data trusted? In microelectronics fabrication – Yes (i.e., Class 100 or 1000 or 100,000 clean rooms.) In Dive Shops or Military diving – No! First of all, this requires a 100x microscopic examination. Secondly, even at 100x it is difficult to see a 2 micron particle (especially a sand particle on a white filter) unless you examine all the filters in both white and black light, or unless the 2 micron particles are in groups. Thirdly, it should be done in a Class 1000 hood or the analysts are kidding themselves.

The field of view at 100x is very small and examining every field is very tedious. Since time is money, this test would easily be the most expensive test a laboratory could run. The only way to make it less expensive is to do a statistical count. This means examining only a certain number of fields of view and multiplying it by the area per field of view. Sound ok? Sorry, that is not allowed because this only works if the airflow is uniform over the entire filter face. Statistical counting is impossible in all compressed air sampling equipment (even ours).

There is a more compelling reason for questioning any particle size specification in compressor air. The reason is common sense obvious. Dive shops, Fire Departments and even the highly technical deep diving Navy all sample the real world. Remember we are not talking about double door, positive pressure Class 100 Clean Rooms. Breathing air sampling is done with equipment that is not manufactured, assembled or used in clean rooms. I have worked with many air kits, and most arrive at the dive shop with enough debris to be seen on a cotton swab.

At least two air sampling kits pass the air through a soft plastic hose before it reaches the filter. At least one lab includes a wire to “clean out the needles”. So, if a Nitrox sample “fails” because of particulate, did it come from your system, the room you were standing in when you assembled the kit, the kit itself, or just during handling of the filter in the laboratory? There is no way to tell; it could even be all of the above at the same time.

The only suggestion I can make to avoid the particle size issue is to rely on a more realistic specification: the US Navy’s specification for Oxygen Compatible air. Certainly the US Navy is more likely to have the assets to investigate criteria than would be the case for a professional organization. (See Specifications.)

EXPERTS: EVERYONE IS AN EXPERT
What does that mean exactly? Recently we were visited by a member of the California Department of Health Services. It was our routine on-site certification inspection. That person was supposed to be an expert in the Environmental analysis field. During his visit, we discovered that he knew less about nearly every aspect of analytical chemistry than some of my newest employees. The Navy has technical programs that are staffed by chemists that are billed as experts. Boy, can I tell you some stories about that – but I won’t.

So, when I hear that an air analysis lab believes that it is expert, my ears perk up. You see, there are very few true experts in this field. To be an expert in sampling and analysis of compressor air, one needs to literally get their feet wet. They need to spend some time sampling in the engine room of a sub-tender where the compressors are installed, or aboard crowded dive boats with integrated HP flasks and LP hose lines. They need to understand why a compressor is failing, and have empathy with the Navy Master Diver who will not tolerate any more dirty fittings in the air sampling kit he has to use.

They need to try different filters in their air sampling kit to note the performance difference. They need to hear the roar of air coming from a 1500 SCFM LP compressor and realize the inadequacy of the sample they just took at 3.5 SCFM. They need to see a plug of rusty water shoot across the room when an air system is turned on at normal flow just after your test showed that the air was clean. And then they need to do some rigorous research and problem solving on analytical methods in the laboratory. But, that’s another story – a long one.