Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Court Orders Breathalyzer Code Opened, Reveals Mess

timothy posted more than 5 years ago | from the take-a-sober-look-at-this dept.

The Courts 707

Death Metal writes with an excerpt from the website of defense attorney Evan Levow: "After two years of attempting to get the computer based source code for the Alcotest 7110 MKIII-C, defense counsel in State v. Chun were successful in obtaining the code, and had it analyzed by Base One Technologies, Inc. By making itself a party to the litigation after the oral arguments in April, Draeger subjected itself to the Supreme Court's directive that Draeger ultimately provide the source code to the defendants' software analysis house, Base One. ... Draeger reviewed the code, as well, through its software house, SysTest Labs, which agreed with Base One, that the patchwork code that makes up the 7110 is not written well, nor is it written to any defined coding standard. SysTest said, 'The Alcotest NJ3.11 source code appears to have evolved over numerous transitions and versioning, which is responsible for cyclomatic complexity.'" Bruce Schneier comments on the same report and neatly summarizes the take-away lesson: "'You can't look at our code because we don't want you to' simply isn't good enough."

cancel ×

707 comments

Sorry! There are no comments related to the filter you selected.

But does it work? (4, Insightful)

will this name work (1548057) | more than 5 years ago | (#27955325)

Poorly written code is one thing, but does it ultimately work?

Re:But does it work? (4, Insightful)

Jason1729 (561790) | more than 5 years ago | (#27955359)

Does it matter? The real question is "Can a prosecutor convince a computer illiterate judge beyond reasonable doubt that it does ultimately work?".

Re:But does it work? (5, Insightful)

Yold (473518) | more than 5 years ago | (#27955433)

I read the report earlier, and there are some very valid issues with the source. The first is that in incorrectly averages readings taken, assigning more weight to the first reading than the subsequent ones. It also has a buffer overflow issue, where an array is being written past its end, and even if this results in an error, it goes unreported.

You would have to be a fricken moron not to have a problem with mis-averaging, however in my experiences with law-people, they can be even worse than PHBs.

Re:But does it work? (4, Insightful)

internerdj (1319281) | more than 5 years ago | (#27955779)

Also it looks like their out of range error scheme was to set it to the closest legal value and report it if it was recurring and continuous. Assume for a moment you took a test right after the last good reading, you took 32 samples. It would only report an error if all 32 samples failed. Otherwise 31 of the 32 will report the maximum legal extreme closest to that reading. Couple that with the fact that the averages were taken incorrectly, this isn't just reasonable doubt it is worse than using a RNG to find if they are drunk.

Re:But does it work? (5, Informative)

Anonymous Coward | more than 5 years ago | (#27955909)

>> assigning more weight to the first reading than the subsequent ones.

It seems to apply more weight to later readings:

where a1=1, b1=2, c1=3, d1=4
(A1+B1+C1+D1)/4 = 2.5 (the correct average)
    and
(((((A1+B1)/2)+C1)/2)+D1)/2 = 3.125

Re:But does it work? (5, Insightful)

fracai (796392) | more than 5 years ago | (#27955993)

Presuming it's the same summary that I read, it contained a mistake.

Readings are Not Averaged Correctly: When the software takes a series of readings, it first averages the first two readings. Then, it averages the third reading with the average just computed. Then the fourth reading is averaged with the new average, and so on. There is no comment or note detailing a reason for this calculation, which would cause the first reading to have more weight than successive readings.

This actually places more weight on the final reading, not the first.

Re:But does it work? (1)

moderatorrater (1095745) | more than 5 years ago | (#27955441)

probably

Re:But does it work? (4, Insightful)

MozeeToby (1163751) | more than 5 years ago | (#27955675)

I'd be more interested in their test plan and test results than their source code if I were trying to convince a computer illiterate judge of something. Find a missing test case or an uncovered corner condition and you might have a decent case, code that doesn't pass static analysis and is ugly... well that pretty much defines 99% of the code out there.

Re:But does it work? (4, Insightful)

plague3106 (71849) | more than 5 years ago | (#27955763)

code that doesn't pass static analysis and is ugly... well that pretty much defines 99% of the code out there.

It's more than ugly, it's difficult to maintain. Also, this point is largely irrelevent; 99% of the code out there isn't spitting out a number that says you're guilty of a serious offense.

Re:But does it work? (4, Interesting)

MozeeToby (1163751) | more than 5 years ago | (#27956039)

No, but some no trivial amount of code is running the x-ray machine at the dentist, processing my credit card, managing my fuel injection, saving my thesis paper, and timing stoplights throughout my city.

We trust our lives and livelihoods to shitty code every day and the plain fact of the matter is that shitty code usually works. As programmers we like to think of ourselves as artists; creating a master piece of perfectly engineered code. In reality, all projects face budget and time constraints, most projects have legacy code which is hard to maintain, and most teams have at least one guy who just doesn't get it.

If the code works, and you can show empirically that the code works, that is proven beyond a reasonable doubt it my opinion. Not beyond any doubt, but that isn't the standard that our justice system is based upon.

Re:But does it work? (3, Insightful)

wiredlogic (135348) | more than 5 years ago | (#27955765)

Regardless of the state of the code no breathalyzer truly "works". None of them can directly detect blood alcohol content. All they do is use a proxy to estimate using the reaction products from your breath. These devices are wholly unscientific. There is no possible way they can derive a credible estimate with a precision of 0.001% or even 0.01%. There is no accounting for body size, type, or metabolic rate. Furthermore these devices can be triggered by more than just ethanol. Chocolate is reported to cause a false reading as well as other common foods. This is why one should always refuse a breathalyzer test even if you haven't been drinking. There are too many chances for a false positive to risk it.

Re:But does it work? (2, Insightful)

Hognoxious (631665) | more than 5 years ago | (#27955889)

There is no accounting for body size, type,

There's no need to. It measures a concentration (per unit volume), not how many beers you drank.

or metabolic rate.

Irrelevant. Just because you'll sober up quicker doesn't mean you aren't drunk now.

Try not to hit anybody on your way home.

Re:But does it work? (2, Insightful)

Yold (473518) | more than 5 years ago | (#27955893)

In Minnesota (and other states), it is a crime to refuse a roadside breathalyzer test due to "implied consent" laws.

Re:But does it work? (4, Informative)

Yold (473518) | more than 5 years ago | (#27955949)

correction, you may refuse a roadside breath test, but not one at the police station.

Re:But does it work? (2, Insightful)

DnemoniX (31461) | more than 5 years ago | (#27955879)

Well the problem with calculating the averages should honestly be enough to get this tossed. The defense can put up an exhibit with a set of numbers using the flawed methodology which shows a person to be over the limit. Then call an expert witness with a math degree, or an accountant for that matter. Show that the average when calculated normally is below the the legal limit. Even better is if you can show that the machine has calculated an average that falls below the legal limit but should have been above. That would show the device to be a public safety hazard for clearing persons who are actually impaired.

Re:But does it work? (1)

al0ha (1262684) | more than 5 years ago | (#27955995)

Being a computer expert, or an expert about anything other than the law, is not the job of a prosecutor, judge or defense attorney.
Thus I argue it does matter and it matters that independent experts can review the device and make conclusions to be presented in court.

Re:But does it work? (4, Insightful)

gd2shoe (747932) | more than 5 years ago | (#27955437)

Good question, but it needs to be reworded. Does it always work for all inputs?

Also important, if it's a poorly written mess, why is the company claiming that it works? I see no indication that they've done due diligence for a device used to convict people. Just because they've never observed it to fail, doesn't mean a thing.

Re:But does it work? (3, Insightful)

Volante3192 (953645) | more than 5 years ago | (#27955789)

If I read the report right, they coded the thing to never actually fail in the first place. It'll always return what can be passed off as a legitimate answer.

Re:But does it work? (3, Insightful)

digitalunity (19107) | more than 5 years ago | (#27955809)

Looks like the answer is no. It's a black box that doesn't report internal errors except when it can't ultimately decide on an answer.

The source code is useful only for showing the machines can be unreliable in certain circumstances, but unless he has substantiating evidence to show it gave an incorrect result he is unlikely to prevail.

Example: Guy blows .09 after drinking 2 beers. He might have a case that the machine was wrong. Example 2: Guy drinks 8 beers and blows .18. Machine might be wrong, but even if it was off by a bit due to rounding averages, he's still guilty as sin.

Sucks, but that's just the way the law looks at it.

Someone mentioned earlier that the weighting of samples under repeat tests give weight to the first blow, which is a big red flag. The initial blow is probably the sample most likely to be contaminated by liquid from the mouth which will skew the reading dramatically, leading to higher BAC's than actuality. If someone blew a .12 and then a .07 on the same machine, he could be found guilty but it's possible the second sample is more accurate.

Re:But does it work? (5, Informative)

geekgirlandrea (1148779) | more than 5 years ago | (#27955447)

Read the article. The code in question, among other things, calculates an arithmetic mean of a sequence of values by successively averaging each value with the mean of all the previous ones, and reduces 12 bits of precision coming from the hardware sensor to 4 for some unspecified but undoubtedly stupid reason.

Re:But does it work? (2, Insightful)

Man On Pink Corner (1089867) | more than 5 years ago | (#27956051)

The code in question, among other things, calculates an arithmetic mean of a sequence of values by successively averaging each value with the mean of all the previous ones, and reduces 12 bits of precision coming from the hardware sensor to 4 for some unspecified but undoubtedly stupid reason.

Well, it's not hard to imagine why they throw away all those bits. Prospective LEO customer: "Wow, this thing never gives the same reading twice. How am I supposed to secure convictions with numbers this flaky?"

The averaging thing, on the other hand... that's just a dumbass bug. One that's probably wrecked a few peoples' lives.

Re:But does it work? (4, Insightful)

mea37 (1201159) | more than 5 years ago | (#27955459)

My first thought as well.

Of course, with poorly written code, it's hard to show whether or not the code ultimately works by examination of the code.

Then again, proving that the code works (which should be the standard when the code is analyzed in court) by code examination is very difficult even for well-written code.

Perhaps a better approach would be documented, repeatable testing of the device. When I challenge a radar gun, I get to ask about its calibration documents, but I don't think I get to debate the blueprints from which it was built.

My personal opinion - and before getting on an "innocent until proven guilty" kick bear in mind that I'm not a part of the court system in this case - is that the defense realizes that almost all software systems look awful and are trying to game their way out of a conviction they've probably earned.

That said, if for no other reason then to eliminate such gaming, there should be standards for testing and documenting the proper function of these devices. Any device that can't be calibrated and tested with sufficient certainty should be banned from use as evidence in court. If the device passes the test, then exactly how it does it shouldn't really matter.

Re:But does it work? (4, Insightful)

vertinox (846076) | more than 5 years ago | (#27955731)

Of course, with poorly written code, it's hard to show whether or not the code ultimately works by examination of the code.

Of course it works because it gives an end result instead of an error message.

The question every should ask is "Does it work accurately?" or "Does poorly written code skew the results?"

Can the defense prove that the code was written so haphazardly that it ignores some data or does it round incorrectly like Excel does? These things do and can happen with sloppy code.

That said, if the code is just poorly commended and indented correctly (*wink*) but does the math right and makes sure there isn't a sampling or rounding problem, then it isn't a problem.

Re:But does it work? (5, Insightful)

Carnildo (712617) | more than 5 years ago | (#27955871)

Perhaps a better approach would be documented, repeatable testing of the device. When I challenge a radar gun, I get to ask about its calibration documents, but I don't think I get to debate the blueprints from which it was built.

Calibration and testing won't reveal all the edge cases that might cause errors. Consider a radar gun designed to take the average of five samples. You've got a car moving away from you at 70 MPH, and a duck flies into the beam for one sample, moving towards you at 5 MPH. This gives the following five samples:

70 70 70 -5 70

I can see a way that badly-written code would turn that into an average speed of 106 MPH (storing a signed char as an unsigned char, which would turn the -5 into a 251), and yet it would pass calibration and every test someone's likely to perform.

No. (5, Informative)

SanityInAnarchy (655584) | more than 5 years ago | (#27955525)

Just read Schneier's comments. He cites some of the more important things:

Readings are Not Averaged Correctly: When the software takes a series of readings, it first averages the first two readings. Then, it averages the third reading with the average just computed... There is no comment or note detailing a reason for this calculation, which would cause the first reading to have more weight than successive readings.

That alone should be enough -- the readings are not averaged correctly. But it goes on:

The A/D converters measuring the IR readings and the fuel cell readings can produce values between 0 and 4095. However, the software divides the final average(s) by 256, meaning the final result can only have 16 values to represent the five-volt range (or less), or, represent the range of alcohol readings possible. This is a loss of precision in the data; of a possible twelve bits of information, only four bits are used. Further, because of an attribute in the IR calculations, the result value is further divided in half. This means that only 8 values are possible for the IR detection, and this is compared against the 16 values of the fuel cell.

So we know it's buggy and inaccurate, to a moronic degree. If that wasn't enough:

Catastrophic Error Detection Is Disabled: An interrupt that detects that the microprocessor is trying to execute an illegal instruction is disabled, meaning that the Alcotest software could appear to run correctly while executing wild branches or invalid code for a period of time. Other interrupts ignored are the Computer Operating Property (a watchdog timer), and the Software Interrupt.

So, basically, it's designed to always return some value, even if it's wildly inaccurate, and even if the software is executing garbage at the time.

In other words: It appears to be a very low-level equivalent of Visual Basic's "on error resume next".

Whiskey. Tango. Foxtrot.

So to answer your question: No, it does not work. Even if it did somehow work, there's obviously an unacceptably poor level of quality control here.

Re:No. (-1, Flamebait)

91degrees (207121) | more than 5 years ago | (#27955603)

So we know it's buggy and inaccurate, to a moronic degree. If that wasn't enough:

Not really. It actually ultimately has 3 bits more than are needed. What they need from the device is for it to tell you whether you're above or below the legal limit.

The design sucks, but it's still possible that it never provides a false positive.

Re:No. (2, Informative)

spun (1352) | more than 5 years ago | (#27955915)

Sure, one bit would be enough to make a pass/fail decision. But they throw away info BEFORE making that determination. You can make a determination and round it down to one bit, but you can't round down to one bit and then make an accurate determination, this is an analog sensing device we are talking about. Throw away everything but one bit, and you don't have a yes/no on the legal limit, you have 'above 2.5v, or below 2.5v.' What's the legal limit, translated into volts, hmmm?

Re:No. (2, Insightful)

FatAlb3rt (533682) | more than 5 years ago | (#27955969)

Do you really think that you should receive the same consideration as a guy that's 3x over the limit? Blowing .08 and .18 are quite different in terms of state of mind.

Definitely possible that there's no false negatives, but for a device that can have such this level of an impact on someone's life, you'd think it would be held to a higher standard.

Re:No. (5, Insightful)

Ohio Calvinist (895750) | more than 5 years ago | (#27955911)

The problem in a lot of states is that .01 can make a huge difference between a DUI, a DUI with a "high BAC kicker", a wet-reckless, or nothing at all. It has to be accurate to at least a few 9's or for those "on the bubble" cases do have a severe level of doubt. Because driving with a .07 is not illegal (for the most part), but .08 is. The question in court is not "were you drinking tonight", but "how much did you drink" which is a very specific very objective, very deturminable piece of information.

As states lower their legal limits to the point where they intersect with non-impaired drinking drivers, especially with a .01 or more margin of error, you're going to get a lot of overzealous cops in cities with revenue shortfalls taking innocent people in for DUIs and hopefully more and more of these "border cases" will bring these devices into question more than the over-the-top blacking out, pissing his pants multiple-offender does in court.

Re:No. (3, Insightful)

Lord Ender (156273) | more than 5 years ago | (#27956053)

In embedded systems programming, it is common practice to disable interrupts if they are not used. It is certainly possible that this app simply does not need to handle these interrupts, whether they are enabled or not.

It is also possible that the other flaws mentioned, which clearly reduce accuracy, do not do so sufficiently to change the outcome in a meaningful way.

The problem with drunk driving law is not primarily one of testing. It is that it presumes someone is incapable of driving with even trace amounts of alcohol, while treating other forms of more dangerous driving (such as driving while texting or on the phone) as being OK or far far less severe.

The way the laws themselves are written is a horrible miscarriage of justice. This is the result of the perverse and hypocritical views of MADD and its ilk, the bastard children of the prohibition movement.

Re:No. (1)

Bananenrepublik (49759) | more than 5 years ago | (#27956055)

Readings are Not Averaged Correctly: When the software takes a series of readings, it first averages the first two readings. Then, it averages the third reading with the average just computed... There is no comment or note detailing a reason for this calculation, which would cause the first reading to have more weight than successive readings.

This is incorrect:
.5 * (1 + 1) = 1
.5 * (.5*(1 + 1) + 1) = .5*(1 + 1)= 1
.5 * (.5*(.5*(1 + 1) + 1) + 1) = 1
...
.5 * (1 + 99) = 50

no matter how long the ellipsis. IOW the last measurement has the highest weight if indeed the algorithm works as described. Of course that's still wrong.

Re:No. (0)

Anonymous Coward | more than 5 years ago | (#27956083)

"would cause the first reading to have more weight" Not correct. Later readings would have more weight. For example, after 3 readings (A, B, & C), the result is 0.25A + 0.25B + 0.5C. The last reading always contributes half the value of the average.

It has to be proven to work (1)

Chmcginn (201645) | more than 5 years ago | (#27955591)

If a breathalyzer was a person, it would be required to be schooled and licensed in the appropriate field of study to be used as evidence at a trial. I might have an amateur interest in ballistics, but I'm not qualified to testify about it at a trial, regardless of the fact that I'm possibly more familiar with my particular firearm than the expert witness hired.

Re:But does it work? No. (1)

Bellegante (1519683) | more than 5 years ago | (#27955813)

It would have to be written correctly to work, wouldn't it? even if the code runs flawlessly: It doesn't average correctly. It tries several times to analyze a sample, but then averages them incorrectly, point 2 FTFA. They can program, but failed basic math.

Re:But does it work? (1)

Bill, Shooter of Bul (629286) | more than 5 years ago | (#27955881)

Not accurately enough to be used in a court of law.

Re:But does it work? (1)

JWSmythe (446288) | more than 5 years ago | (#27955921)

    I've read through some really shitty code before. Sometimes you can't determine if it should work right or not, but when you start spotting flaws of a significant magnitude, you can assume the result is wrong.

    Since this was never intended for release, and they got paid for it, as long as it gave something, they were happy. Now it's come to bite them in the ass. Ha!

    I try to write like there will be an audience someday, even if it's just me reading my own code years later.

   

Re:But does it work? (1)

Reece400 (584378) | more than 5 years ago | (#27956015)

By the sounds of it, it works so long as all the hardware is fine and new. However if any electronics malfunction it either won't know about it, or ignore it unless it hasppens 32 times in a row! Meaning a broken one could give erroneos results every other test without ever raising any red flag!

Lint is crap (1, Insightful)

xenocide2 (231786) | more than 5 years ago | (#27955333)

Lint, as a static code analyzer, is bound to have false positives. More so in embedded systems, where you're dealing with registers and occasionally "violating" type safety where no type adequately exists. It's really not super surprising that 60 percent of the code is reported by Lint.

Re:Lint is crap (4, Interesting)

TigerNut (718742) | more than 5 years ago | (#27955741)

The thing is that probably 95% of the Lint reports could have been fixed by the code designers, just by making appropriate declarations or a bit of type casting. The fact that 60% of the source is reported by Lint, indicates that the designers never bothered to do any kind of static code checking or to clean up warnings, and that points to a lack of care during development and testing.

At a previous job we had to buy a third-party driver for an embedded PCMCIA controller. The software vendor delivered code that (the first time around) produced about 1200 lines of warnings when we compiled it. We queried them about it and they responded that "we don't compile with warning output enabled". Our reply to them was that our coding standard was that the compile would fail on warnings, and we wouldn't accept their code unless they fixed all the warnings... they cleaned up their act, and fixed a couple of previously unresolved problems in the process.

Re:Lint is crap (0)

Anonymous Coward | more than 5 years ago | (#27955873)

Any static analysis is going to have false positives. There are better tools available than Lint, but even those tend to have enough false positives to make the percentage look fairly horrible.

Just because it cries wolf a lot doesn't mean that it's not occasionally correct.

Coding Standard (1, Interesting)

Jack9 (11421) | more than 5 years ago | (#27955357)

nor is it written to any defined coding standard

Er, why would it need or be expected to be? It's a commercial product. I don't think most bank websites are "coded" to any specific standard either.

Re:Coding Standard (0)

Anonymous Coward | more than 5 years ago | (#27955393)

To look at a different kind of code, how does US legal code work? It is a patchwork which is only rarely organized into a coherent structure, but it usually gets us through our daily routine. Though I guess we would benefit from open source legal code [metagovernment.org] .

Re:Coding Standard (4, Insightful)

kailoran (887304) | more than 5 years ago | (#27955401)

Because the output is used as evidence in court?

And does it work? (1)

SterlingSylver (1122973) | more than 5 years ago | (#27955539)

So I can't tell if this analysis of the code indicates that the breathalizers in question are somehow flawed. Perhaps they're coded inelegantly or poorly, but do they actually spit out inaccurate numbers?

Re:And does it work? (3, Insightful)

legirons (809082) | more than 5 years ago | (#27956059)

Perhaps they're coded inelegantly or poorly, but do they actually spit out inaccurate numbers?

Irrelevant - the test is: do they always spit-out provably correct numbers?

Re:Coding Standard (3, Insightful)

SirGeek (120712) | more than 5 years ago | (#27955541)

Er, why would it need or be expected to be? It's a commercial product. I don't think most bank websites are "coded" to any specific standard either.

From the article:

1. The Alcotest Software Would Not Pass U.S. Industry Standards for Software Development and Testing: The program presented shows ample evidence of incomplete design, incomplete verification of design, and incomplete "white box" and "black box" testing. Therefore the software has to be considered unreliable and untested, and in several cases it does not meet stated requirements. The planning and documentation of the design is haphazard. Sections of the original code and modified code show evidence of using an experimental approach to coding, or use what is best described as the "trial and error" method. Several sections are marked as "temporary, for now". Other sections were added to existing modules or inserted in a code stream, leading to a patchwork design and coding style.

Ok. Would you want to have something that can cause you to get convicted because it wasn't documented or even tested fully - ("Oh, Crap. That constant should have been 0.001, not 0.01. Ooops. Blood Alcohol level was 0.008, not .08. Sorry !")

Common sense (if it WERE common) should indicate that there should be full tests for a wide range of values performed with the written tests and expected values verified and available to prove that the device/software actually does detect the proper levels of alcohol.

Re:Coding Standard (2, Interesting)

0100010001010011 (652467) | more than 5 years ago | (#27955597)

Some of this stuff is elementary math.

2. Readings are Not Averaged Correctly: When the software takes a series of readings, it first averages the first two readings. Then, it averages the third reading with the average just computed. Then the fourth reading is averaged with the new average, and so on .... the comments say that the values should be averaged, and they are not.

It's been a while but didn't the teacher in 5th grade show you why that wouldn't work?

Or how about this:

The A/D converters measuring the IR readings and the fuel cell readings can produce values between 0 and 4095. However, the software divides the final average(s) by 256, meaning the final result can only have 16 values to represent the five-volt range (or less), or, represent the range of alcohol readings possible.

Who the hell didn't pay attention in the A/D quantization error in controls class?

I don't want to fill my whole comment with copy and paste from TFA, but not only is this a code standard issue. It's just plain stupidity. Error checking, out of range checking all sound like something a first year programmer should have gotten right.

I can't say all my MATlab and CANape scripts look pretty or are up to any coding standards, but I try to at least get the basic shit right.

Re:Coding Standard (0)

Anonymous Coward | more than 5 years ago | (#27955645)

I think you are think of trade standards (e.g., Mizra, DO-178B, DEFSTAND, etc.). I took it as they had no standards and conventions they followed. Any piece of code of significant size SHOULD follow some convention for conformity and consistency. That's just good coding practice. Every project or software shop MUST have a standards and conventions document that explains what processes are to be following in developing code. That is software engineering 101.

Re:Coding Standard (1)

Yold (473518) | more than 5 years ago | (#27955657)

Errors crashing a bank website result in loss of dollars, a false negative (drunk but they let you go) reading from a Breathalyzer may result in a drunk driver killing someone.

U.S. Military, FAA, and other government organizations have coding standards for commercial products, why should a device intended to gather evidence and promote public safety be unregulated? Especially in this case, where the device can unrecoverable fail due to a logic-error, and not report that such an error occurred?

Don't forget the false positives (2, Insightful)

Chmcginn (201645) | more than 5 years ago | (#27955827)

In general, I was under the impression that the standard for criminal cases were weighted heavily to reject any technique, evidence, or device that had any appreciable chance of a false positive.

Re:Coding Standard (1)

Reece400 (584378) | more than 5 years ago | (#27956073)

<sarcasm>Well, depending on whose money is lost with the website problems, I could see a lot of bankers in fear for their lives.</sarcasm>

Re:Coding Standard (0)

Anonymous Coward | more than 5 years ago | (#27956047)

Wow, please tell me the products you work on so I can be sure to avoid them. Not that you're wrong about whether it's expected, but that rather than say something like "The current casualness of the software engineering for systems playing huge roles in our lives - voting, medical software, breathalyzers - is really disturbing." you make it sound like it's okay because most everyone else writes without standards.

When I hear stuff like this, it makes me more heavily in favor of turning software engineering into a legitimate profession with testing, standards of conduct, and training requirements such as coursework or internships. Unlike with doctors, I think we should allow unlicensed software engineers but all products would need to be clearly labeled whether or not they were made by professionals (which would effectively hold them responsible if standards were not met). I think it would be good for everyone, and it'd help companies make a clear decision about whether they value quality in their software.

Ballmer Peak (2, Funny)

Anonymous Coward | more than 5 years ago | (#27955375)

Clearly their programmers were not drunk enough when making this. Or, they were too drunk.

Re:Ballmer Peak (3, Funny)

Anonymous Coward | more than 5 years ago | (#27955465)

Clearly their programmers were not drunk enough when making this. Or, they were too drunk.

How else would you suggest they test whether or not it works? Huh, smartguy?

Code (5, Insightful)

Quiet_Desperation (858215) | more than 5 years ago | (#27955389)

not written well, nor is it written to any defined coding standard

Ah, so it's like most of the code in the world.

Re:Code (1, Insightful)

Anonymous Coward | more than 5 years ago | (#27956031)

except that this code is used to convict people.

Good! (4, Insightful)

SanityInAnarchy (655584) | more than 5 years ago | (#27955409)

Ok, I'm not happy that some people almost certainly were measured inaccurately by these things. I'm not happy that this company was allowed to pull this kind of shit -- when you do government contracting, the government should own what you do.

However, I am very glad that the precedent has been set.

And I am especially glad that not only is there precedent, but there's a real live example of why we need this stuff to be open.

Re:Good! (5, Insightful)

Red Flayer (890720) | more than 5 years ago | (#27955455)

when you do government contracting, the government should own what you do

But they weren't doing government contracting. The produced a good that was purchased by the government. There's a very big difference.

The key here is not that the government, or anyone, should own what they produced -- it's that when what they produced is used to convict someone, that person has the right to examine the methods used.

It's not about openness, at all. It's about the right to a fair trial; openness is just a side effect.

Watch those comments... (2, Funny)

tcopeland (32225) | more than 5 years ago | (#27955423)

...from the article:

Several sections are marked as "temporary, for now".

So, make sure to strip out those TODOs before checking in the code. Bah!

No surprise (4, Insightful)

infinite9 (319274) | more than 5 years ago | (#27955425)

80% of the code in business fits this description. With 20 year old legacy code written by 50 consultants, then upgraded in India, then ported from one platform to another to another, and a database engine switch or two. Code gets senile. What do they expect? Good thing we're all just commodities... human lego bricks easily replaced with cheaper plastic.

Re:No surprise (1)

gEvil (beta) (945888) | more than 5 years ago | (#27955523)

Good thing we're all just commodities... human lego bricks easily replaced with cheaper plastic.

Speak for yourself. I'm a Duplo.

Re:No surprise (1)

guruevi (827432) | more than 5 years ago | (#27955621)

At some point it's better to just rewrite the thing from the ground up. Especially once you start moving platforms (if your code is architecture specific enough like a kernel) or definitely when you start moving between languages (when you start integrating somebody else C code in what used to be another one's Perl script).

Re:No surprise (1)

infinite9 (319274) | more than 5 years ago | (#27955863)

At some point it's better to just rewrite the thing from the ground up. Especially once you start moving platforms (if your code is architecture specific enough like a kernel) or definitely when you start moving between languages (when you start integrating somebody else C code in what used to be another one's Perl script).

This is obvious to developers, but a completely alien idea to business users or executives. To them, that 20 year old crap application is a valuable asset. And to suggest that they throw it out and replace it is like suggesting that they burn down their office building and buy a new one.

I'm really getting jaded with this whole IT thing. I'm seriously considering moving to a small town and doing something completely different. Maybe I'll be a mail man, or open my own auto shop.

Re:No surprise (1)

Chmcginn (201645) | more than 5 years ago | (#27955985)

I'm really getting jaded with this whole IT thing. I'm seriously considering moving to a small town and doing something completely different. Maybe I'll be a mail man, or open my own auto shop.

I decided IT was for the birds, and that doing more hands-on electrical work was a much better idea. And then I got picked to troubleshoot PLC systems...

If you want to see some god-awful code, try working on fifteen year old monitoring system for a thirty year old generator.

Re:No surprise (0)

Anonymous Coward | more than 5 years ago | (#27955947)

80% of the code in business fits this description.

Yeah, but how much of that code's entire purpose is to decide if someone is a criminal?

The precedent that needs to be set here is any code used to produce evidence must be open for inspection. This should be a straight corollary of "justice must be seen to be done".

asus notebook (0)

Anonymous Coward | more than 5 years ago | (#27955463)

I don't think most bank websites are "coded" to any specific standard either.

http://www.myshop.com.tr

not written to a coding standard? (2, Insightful)

AliasMarlowe (1042386) | more than 5 years ago | (#27955471)

Just because code is not written to some official standard does not mean it is guaranteed to be buggy. Undisciplined coding is as bad as undisciplined specifications - results can indeed be ugly. It is preferable if the coders follow good practices, and there ideally would be a clear system for specifying program behaviour in testable ways. It is easier to produce good code with robust behaviour if good practices are followed from design through coding to testing and documentation, but it is not impossible to achieve good results in other ways also.
Did they find any coding bugs, or did they just criticize the approach to coding?

Re:not written to a coding standard? (0)

Anonymous Coward | more than 5 years ago | (#27955683)

Surprisingly, the lack of standards is just one of several stated criticisms, and if you RTFA you'd see more details.

Re:not written to a coding standard? (5, Insightful)

SanityInAnarchy (655584) | more than 5 years ago | (#27955729)

Did they find any coding bugs,

Yes. RTFA.

2. Readings are Not Averaged Correctly: When the software takes a series of readings, it first averages the first two readings. Then, it averages the third reading with the average just computed.

There you go. It's also inaccurate:

The A/D converters measuring the IR readings and the fuel cell readings can produce values between 0 and 4095. However, the software divides the final average(s) by 256... Further, because of an attribute in the IR calculations, the result value is further divided in half. This means that only 8 values are possible for the IR detection...

And, if there were a catastrophic bug, you wouldn't know it, you'd just keep getting readings:

An interrupt that detects that the microprocessor is trying to execute an illegal instruction is disabled, meaning that the Alcotest software could appear to run correctly while executing wild branches or invalid code for a period of time. Other interrupts ignored are the Computer Operating Property (a watchdog timer), and the Software Interrupt.

This belongs on The Daily WTF.

Re:not written to a coding standard? (1, Informative)

Anonymous Coward | more than 5 years ago | (#27955815)

If you read the points beyond the first one, they indicate some pretty serious problems: All readings are reported relative to a baseline reading taken when the device first turns on, but there are no sanity checks in place for this reading, nor are there attempts to establish an initial average. The code that's supposed to average the user readings doesn't average, but instead heavily weights the first readings. Error handling is available but disabled; there is no way to determine a malfunctioning device from a functioning one. Known bad values are simply clamped to a valid range.

Granted, the thousands of "errors" mentioned in the article are probably mostly lint warnings... While worrying, they are not actual coding errors.

But it stands that this device could report an alcohol level that is higher or lower than detected in various circumstances. And these cases could have been detected or avoided if the software had been written better.

Re:not written to a coding standard? (1)

Todd Knarr (15451) | more than 5 years ago | (#27955941)

Well, there are two that raise questions with me immediately. First, where the specifications called for 4 readings to be averaged, the code uses an exponential moving average instead. That's bad because the moving average weights the last sample at 50% of the final result, while the purpose of averaging is to give all samples equal weight so the inherent errors in each sample will (usually) cancel each other somewhat out making the error of the average less than the error of each individual sample. At the very least it indicates the programmers didn't understand the problem they were being asked to solve. Second, the reduction of the sample resolution from 12 bits (4096 values) to 3 bits (8 values) is strange. That alone introduces a fairly large error into the results just by squashing 256 distinct sample values into one. That means that a reading of 1 (negligible) on the sensor would yield the same result on the display as a reading of 511 (not negligible anymore), and that the device can only report 8 BAC readings to cover the entire range it purports to measure.

Combined with the large number of warnings from lint, that kind of thing is exactly what'd make me very cautions about a codebase. Lots of small bad habits in the code combined with obvious indications the programmers either didn't quite understand what they were doing or weren't at all careful about making sure what they did was what they intended spells waist-deep morass of bugs to me.

Perhaps, but... (1)

91degrees (207121) | more than 5 years ago | (#27955497)

What is the typical variance of the readings? Does the division use the simple built-in rounding down or have they added rounding? What range of values can the breathalizer represent? Is this device used as the only device to determine intoxication level?

Ultimately the question boils down to has anyone been convicted directly as a result of one of these when their blood alcohol level was within legal limits?

Windows (1, Interesting)

Anonymous Coward | more than 5 years ago | (#27955501)

And the kicker? The new version of the breath tester runs Windows:

While Draeger's counsel claims that the "The Alcotest [7110] is the single best microprocessor-driven evidential breath tester on the market", Draeger has already replaced the antiquated 7110 with a newer Windows® based version, the 9510. The computer code in the 7110 is written on an Atari®-styled chip, utilizing fifteen to twenty year old technology in 1970s coding style.

Good luck getting the source code for that one, or analyzing it for bugs.

Re:Windows (1)

retchdog (1319261) | more than 5 years ago | (#27956045)

Uh, does Windows software magically not have source code or something?

Just remember (5, Insightful)

captnbmoore (911895) | more than 5 years ago | (#27955543)

This will not stop the state from using this to make a felon of you.

DUH.... (4, Interesting)

Lumpy (12016) | more than 5 years ago | (#27955549)

If you got your hands on and analyzed the sourcecode to most DVD' players, TV's (Panasonic runs linux!) and other devices that are complex you will discover that in order to ship it earlier the code is an utter mess.

Programmers are not joking when we complain about the "It compiles? Ship it!" statement.

the fault is the Executive staff that refuse to listen to their experts (programmers) and do what they recommend. Instead we get morons that know nothing about programming making unrealistic deadlines and forcing death march coding marathons to give up the mess we have today.

Here's a little jem. (2, Funny)

needs2bfree (1256494) | more than 5 years ago | (#27955561)

I thought it was funny.
12. Defects In Three Out Of Five Lines Of Code: A universal tool in the open-source community, called Lint, was used to analyze the source code written in C. This program uncovers a range of problems from minor to serious problems that can halt or cripple the program operation. This Lint program has been used for many years. It uncovered that there are 3 error lines for every 5 lines of source code in C.

While Draeger's counsel claims that the "The Alcotest [7110] is the single best microprocessor-driven evidential breath tester on the market", Draeger has already replaced the antiquated 7110 with a newer Windows® based version, the 9510.

Atari styled chip? (0)

Anonymous Coward | more than 5 years ago | (#27955639)

What the heck does that mean? A 6502?

Re:Atari styled chip? (1)

sentientbeing (688713) | more than 5 years ago | (#27956067)

No. it means the drunk being tested gets to enter his test result into a high score table

About time (1)

hesaigo999ca (786966) | more than 5 years ago | (#27955653)

About freakin time someone put that stupid test to the test, I always knew them to be sometimes buggy, especially when I know some people to have passed that test, when they could have not even stood up on their own, and yet someone who has no discernable symptoms of being drunk, was blowing way over the limit...for sure there were bugs in their system, all systems have bugs...
if M$ cant program without bugs, why do they think they could?

Cyclomatic Complexity? (0)

Anonymous Coward | more than 5 years ago | (#27955673)

SysTest said, 'The Alcotest NJ3.11 source code appears to have evolved over numerous transitions and versioning, which is responsible for cyclomatic complexity

I'm pretty sure the paths through the program measure the cyclomatic complexity. Not that fact that it has numerous 'transitions' or versioning.

Source Code (1)

Niris (1443675) | more than 5 years ago | (#27955733)

Looked through TFA but don't see the actual code. Just curious if that was released to the public, or if only the reports on it were.

the issue is not just how bad the code looked (1)

Cyrus20 (1345311) | more than 5 years ago | (#27955769)

the issue is not just how bad the code looked it was the crap that was in the code. for something that like what the article is talking about it needs to be accurate. as my programing professor used to say. "crap in crap out"

"Reveals Mess" (1)

Anita Coney (648748) | more than 5 years ago | (#27955795)

Seriously, is anyone actually shocked by this? Their code is protected by copyright, so it's not like anyone can simply take the source code and use it. And it's not like there's some sort of secret in the medical proven fact that, generally speaking, a certain percentage of alcohol in a person's breath indicates a corresponding percentage of alcohol in the same person's blood supply. So the only basis the company had to refuse to release the code was because they knew it was a mess.

Looks like the NJ Supreme Court doesn't care (4, Informative)

kelnos (564113) | more than 5 years ago | (#27955841)

It appears that the NJ Supreme Court wasn't swayed too much [thenewspaper.com] by the source code evaluation. They're planning on reinstating the device with only minor modifications.

Was It Wrong, Though? (1)

darkmeridian (119044) | more than 5 years ago | (#27955847)

The source code was a miserable mess. But I didn't see anywhere that the source code caused the machine to give spurious or inaccurate results. Much ado about nothing?

Re:Was It Wrong, Though? (1)

kelnos (564113) | more than 5 years ago | (#27955913)

This comment [schneier.com] to Schneier's blog post might shed some light on the biggest bug in the code. If a user of the device is in one of the only *eight* ranges that the device recognises, there's a 60% chance they're below the legal limit when it will register as above the limit.

(Of course, that analysis relies on a couple [reasonable] assumptions about the range of the device that may or may not be true.)

Re:Was It Wrong, Though? (1)

xenocide2 (231786) | more than 5 years ago | (#27956037)

You can't tell with just the source code. To say the average is wrong, you'd need to have some idea of what the inputs are, and how they vary over time. For that you'd ideally want the actual device, which is was not made available. It's a bit like being asked to demonstrate if a car works, given only the steering wheel.

The code in question (3, Funny)

Linker3000 (626634) | more than 5 years ago | (#27955857)

10 REM Alky 0.1 A. Coder 2001
20 REM Turn off lights and buzzer
24 POKE 201,0
26 POKE 202,0
28 POKE 53280,0
29 REM Any Breath?
30 IF PEEK(200) = 0 THEN GOTO 30
32 REM Buzzer
33 POKE 53280,1
34 PAUSE(2)
35 POKE 53280,0
36 REM Lights...
40 A = 10 * RND(1)
50 IF A > 5 GOTO 80
60 REM Red light
70 POKE 201,1
75 GOTO 100
76 REM Green Light
80 POKE 202,1
100 PAUSE(3)
120 GOTO 20

What is code? (1)

Sybert42 (1309493) | more than 5 years ago | (#27955861)

Is that like the recipe to the Big Mac? That should be a secret, or we won't get innovation in taste.

stop paying lawyers, pay a programmer (3, Interesting)

bcrowell (177657) | more than 5 years ago | (#27955883)

If I were the manufacturer, at this point I'd say: (1) lawyers are expensive; (2) competent programmers are expensive, but less expensive than lawyers; (3) our business is selling the beathalyzer, not the software, so we gain nothing by keeping the source secret; (4) this publicity is hurting us; (5) let's hire some more competent programmers to clean up the code, and then we can make it public; (6) profit!

This is different from the case of the voting machines. In the case of a voting machine, there are lots of people who might be motivated to hack it, lots of people have access to the machines, and it only takes one compromised machine to throw a close election. If you believe in security by obscurity, then there is at least some logical argument for keeping the voting machine code secret. In the case of the breathalyzer, there's not even that lame argument.

Re:stop paying lawyers, pay a programmer (1)

cdrguru (88047) | more than 5 years ago | (#27956061)

Wrong. All they are selling is the software in the hardware - the hardware can be easily copied by any Chinese company and sold for 1/10th the price they are charging - whatever that is.

It is the software that is the only thing that differentiates the product, and would cost a Chinese company far more to duplicate. It is possible (but unlikely) that they further protected the software through the use of absolutely proprietary hardware that can't easily be duplicated. But the key is that for nearly all electronic devices today it is the software that is worth something and the hardware is meaningless.

As for the company having some sort of "look and feel" patent for their device, US Customs have pretty much proven they will not block the import of infringing products. This means that once the software gets out, if there is any sort of market at all for the devices they will be immediately replaced in the market by cheap Chinese copies. Law enforcement is under the same budget constraints as the rest of the country, so they aren't going to look too carefully at who is manufacturing a device, assuming they would care if it were known. They are interested in the overall cost of the device, and that's about it. So if you offer them a cheap copy at 1/10th the price, they are going to leap on it.

US manufacturer then leaves the market because they can't compete. The Chinese folks can take the code, however bad it might be, and just use it. Oh they might have a programmer make a couple of changes so it doesn't have the original manufacturer's name embedded in it - but I have seen cases where that hasn't even been done. It is probably easier to just avoid the programmer altogether.

Hardware is easily duplicated. Software, once it gets out in the world, is easily stolen. And in any sort of embedded system application there are no protections for the original manufacturer. So they are going to lose.

it just occured to me ... (1)

jsnipy (913480) | more than 5 years ago | (#27955895)

Do they keep booze on hand for unit testing?

Past cases (1)

Kokuyo (549451) | more than 5 years ago | (#27955951)

So... what about all those people who have been found guilty after being tested with those devices? What happens to them?

A pyhrric victory for open source and code review. (4, Insightful)

Darth_brooks (180756) | more than 5 years ago | (#27955953)

The good: This particular breathalyzer has been proven to be the unreliable POS that it apparently is. This unit, and others like it, will finally start being held to a stronger coding standard.

The bad: every sleezeball, ambulance chasing, "call lee free", douchebag of a lawyer will use this case to attack the credibility of any and all breathalyzers made in the past, present, or future, spreading enough FUD to juries everywhere that an unacceptable number of drunken idiots get the God given right to keep their license until they finally end up killing someone.

As a person, I think groups like MADD spend most of their time trying to scare monger politicians into pushing us as close to prohibition as possible. I believe that alcohol can be used responsibly. But I also know that this case is going to result in DUI's getting overturned for people that damn sure don't deserve it. Borderline cases will get knocked down, cases will get thrown out, and the people that broke the law, that did something wrong, will walk out of a court room 'vindicated.' They didn't do anything wrong when they had six beers and drove home, it was that confounded *machine* that *said* they broke the law. The *machine* was busted, ergo they didn't break the law. In short, this case is going to make a lot of O.J. Simpson's. The jury said they didn't commit a crime, so they didn't. No harm no foul. Technicality? Bah! They're as innocent as the sweet baby Jesus.

I'd like to think things will wash out in the end. This case will probably end up making it harder to get off on this particular technicality in the long term. In the short term? Here come the appeals. Maybe the state is partially at fault for buying shoddy equipment. (Or maybe not. Did they do a code review? Do they have the resources to one? Probably not. Did you do a code review of the 3com switch in your server room? Their selection criteria can certainly be questioned, but it probably doesn't change the fact that someone drank enough to blow a .22 then decided to drive home.)

But in the end, the drunks are still going to be drunks. And tomorrow some of them will probably get to file appeals, and some of the ones that shouldn't be on the road, or even in public, will get to slip out of this brand new loophole. I'm not sure that that deserves a cork-popping celebration.

(and yes: We all handle our booze differently. Arbitrary limits that determine "drunk" may or may not be the answer. Hardcore drunks will keep driving even after losing their license. DUI's are as much moneymakers for the States as speeding tickets. Yadda yadda yadda.)

The best news: (1)

internerdj (1319281) | more than 5 years ago | (#27956007)

"While Draeger's counsel claims that the 'The Alcotest [7110] is the single best microprocessor-driven evidential breath tester on the market', Draeger has already replaced the antiquated 7110 with a newer Windows® based version, the 9510."
The new improved version is Windows based...

Bruce Schneier needs a math refresher (0)

serutan (259622) | more than 5 years ago | (#27956027)

From his report:

"When the software takes a series of readings, it first averages the first two readings. Then, it averages the third reading with the average just computed. Then the fourth reading is averaged with the new average, and so on. There is no comment or note detailing a reason for this calculation, which would cause the first reading to have more weight than successive readings."

Wrong, Bruce. The first two readings have the least weight, and each subsequent reading has the same weight as all the previous readings combined.

Average after two readings = (r1 + r2) / 2.
Average after the third = ((r1 + r2) / 2) + r3) / 2.
Average after the fourth = (((r1 + r2) / 2) + r3) / 2) + r4) / 2.

The last average boils down to r1/8 + r2/8 + r3/4 + r4/2.

Makin' good money selling your books, are ya Bruce?

Isn't the average just a running average (1)

Jeff1946 (944062) | more than 5 years ago | (#27956071)

My quick read of the averaging method implies that the average is one half the last reading plus one half the previous average. Just a simple way to do a running average. More importantly what are the error limits using the device. If the limit is 0.08 % and it reads 0.09 what is the +/- bounds? How is it zeroed and spanned will of course be important as well.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?