Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: I came, I saw, I ...

by stevieb (Canon)
on Jul 01, 2017 at 18:18 UTC ( [id://1193999]=note: print w/replies, xml ) Need Help??


in reply to I came, I saw, I ...

Fried one of my test RPis by accidentally putting 5v from an Arduino onto a 3.3v GPIO pin, had a beer, and started over.

Replies are listed 'Best First'.
Re^2: I came, I saw, I ...
by afoken (Chancellor) on Jul 05, 2017 at 05:24 UTC
    Fried one of my test RPis by accidentally putting 5v from an Arduino onto a 3.3v GPIO pin, had a beer, and started over.
    • Fried a small TFT display considered for using in a new product just yesterday, by accidentally swapping +3.3V and GND wires while setting up the test circuit. Damage: About 5 €. Lesson learned: Always measure current, and use a lab supply instead of relying on a development board to supply the device under test. Lesson re-learned: Double check supply pins (see below).
    • Fried a custom prototype controller board on a test rig last year, by accidentally touching a wire connected to a 3.3V signal to the +24V supply line. Most semiconductors on the board were not amused and released the magic smoke. Damage: About 100 €. Lesson learned: Banana plug jumpers should better be isolated. And no, I did not build the test rig. That was the job of the hardware guys. But I wrapped the f**ing jumpers carrying supply voltages in electrical tape after that.
    • Fried my first PC (a 90% compatible XT clone) years ago by connecting the non-standard PSU connector shifted by one pin. +12V from the PSU connected to the +5V line on the board. Magic smoke escaped. Damage: About 500 € in current money. Lesson learned: Double-check the power connectors before switching on. ATX PSU connectors can't be plugged in in a wrong way except by excessive force, so that's clearly a forgotten lesson, as shown above.
    • A decade before that, I talked my dad into measuring the output pulse on the primary winding of an old door-bell transformer while feeding in a DC pulse from a 9V battery on the secondary winding. Using the good oscilloscope in the TV workshop, of course. He should have known better. The pulse killed one of the two input channels of the scope. No magic smoke, but sufficient damage to send in the scope for repair. I don't know how he explained the damage to his boss. And I don't know how much damage that little experiment has done to the scope. But I know for sure the pulse exceeded the max. input voltage of the scope by about one order of magnitude. So I guess it killed a good part of the input amplifier, full of manually selected, custom semiconductors. I think that the damage was about 1000 € in current money. Lessons learned: 10:1 probes aren't sufficient for measuring voltages in the kV range. And feeding DC pulses into unknown transformers is a really bad idea.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

      I was laughing up until I got to the, "using the good oscilloscope" part in the last story, at which point I felt my heart sink in to my stomach, face palmed, and let out a mumbled "oh, no" for you and your dad. On a related note, here's one of those old lame engineering jokes. Do you know why they call it a flyback transformer? Old TV repairmen came up with it because they used to fly back across the room when they accidentally touched one inside an old cathode ray tube television set. :-)

      I have been fortunate enough to not have to explain any damaged/destroyed expensive test equipment to any of my employers... many damaged/destroyed prototypes, once an old oscilloscope probe (a relatively cheap one), but never anything like a whole oscilloscope. I'm so paranoid anytime I hook up something really expensive like a network analyzer I triple check everything before connecting power to the unit under test, and even then I flinch a little when I flip the supply switch.

      Just another Perl hooker - But the purist monks keep telling me I should do it for love, not money.
      Fried a custom prototype .... Damage: About 100 €

      Wow - you got off easy. I'm guessing your employer is enlightened enough to include plenty of spares in the order for prototypes. (My employer's purchasing department will only allow us to order the minimum quantity accepted by the board maker. This usually means we have only 2 or 3 spares.)

      I hope your dad didn't get in too much trouble for blowing the channel on the scope.

      The most expensive equipment I blew up wasn't my fault. The PCB layout engineer changed the orientation of a square micro-controller by 90 degrees but forgot to update the board markets. When a technician installed a socket where the micro would go, she followed the board markings. When I got the board and plugged in the in-circuit emulator, the emulator's interface "pod" belched the magic smoke when I powered the board. That was only about US$2000 to replace.

      A co-worker of mine was once subject to the same "fire works" due the same mistake (by a different PCB engineer), but the damage was US$30k.

        I hope your dad didn't get in too much trouble for blowing the channel on the scope.

        Well, he wasn't fired, and he did not have to pay the damage. In fact, the company was sold later, including his working contract (as usual in Germany at that time). During that time, when TV sets had still a potent high voltage supply and a CRT, a scope could easily be killed by measuring at the wrong place of a TV set with the wrong settings. Nobody is stupid enough to measure the 30 kV at the anode of the CRT, but focus and gate 2 also have a voltage that would be quite unhealty for a scope with a standard probe.

        The most expensive equipment I blew up wasn't my fault. The PCB layout engineer changed the orientation of a square micro-controller by 90 degrees but forgot to update the board markets. When a technician installed a socket where the micro would go, she followed the board markings. When I got the board and plugged in the in-circuit emulator, the emulator's interface "pod" belched the magic smoke when I powered the board. That was only about US$2000 to replace.

        That's the "oh f***ing s**t" moment. Misplacing the ICE connector is also quote common on our current project, but luckily the ICE is quite robust, cheap, and some really wise man has decided to use a connector pinout for the ICE that makes it really hard to kill the ICE or the controller. We would normally use a keyed connector on the PCB, but that exact version required for the ICE is either close to unobtainium or insanely expensive. And compatible solutions are simply too large. So we placed an unkeyed connector and a tiny dot indicating pin 1 on the PCBs. So far, we have killed no ICE, and except for my 24V mistake, no board.

        A co-worker of mine was once subject to the same "fire works" due the same mistake (by a different PCB engineer), but the damage was US$30k.

        And that's the moment when you decide to call it a day, go home, and try again the next day.

        My dad had a similar experience during his first job in the R&D department of a major electronics company. Consumer hardware electronics still was full of tubes and some rare germanium and selenium semiconductors. Industry started first projects with expensive and brand-new silicium semiconductors. Bipolar transistors and diodes, with real wires, packed in metal cans. Not that black, wireless birdseed FET stuff in plastic and ceramic cases with far too many pads and unreadable markings. A single Si transistor cost 100 € or more, in current money, and it had awful specs. Si transistors were still individuals, each with its own set of properties. Finding some pairs with similar behaviour could take a day. One day, one of the hardware engineers came with a sketch of a high-end, high-power, high-frequency, all-transistor amplifier, requiring about 20 of those expensive beasts, all paired, all wired in parallel, with paired resistors. Plus a few more smaller transistors to control the big ones, also paired. About 2 or 3 k€ in modern money, maybe more, just for the parts and a few days for the selection. "The exact spec does not matter, just find similar ones", he said. My dad build the amplifier, connected it to the dummy load (capable of heating away several kW) and a supply, switched on power to start adjusting the amplifier, and just saw all of the transistors die faster that he could switch off. He went to the engineer, confessed that he had just killed hardware worth more than he would earn in two or three months. "Oh well, I kinda expected that would happen. My fault. Damn. I will have to redesign that better." And that's all the engineer said. Buring 3 k€ (in current money) and one or two weeks of work just to test an idea was completely acceptable.

        Looking at my damages over time, I see a quite good trend. The damages get smaller. Let's hope the best ...

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
        Fried a custom prototype .... Damage: About 100 €

        Wow - you got off easy. I'm guessing your employer is enlightened enough to include plenty of spares in the order for prototypes. (My employer's purchasing department will only allow us to order the minimum quantity accepted by the board maker. This usually means we have only 2 or 3 spares.)

        Well, yes and no.

        We had exactly two prototype boards from the first generation, and we modified the hardware quite a bit while integrating the hardware and software. I remember a little "SMD Stonehenge" of four or six SMD resistors they were wrongly connected as pull-downs, not pull-ups. So our hardware expert mounted the resistors vertically, only on the pads of the signal lines, soldered a thin wire to the other ends, and connected that wire to a 3.3V pad somewhere nearby. It looked quite nice, but of course, it was a hack to get stuff working. Tons of other stuff went wrong, especially with a nasty serial bridge chip. Those boards ended in a box somewhere in the storage room, and IIRC, our hardware expert later ripped some spare parts from them to fix other boards.

        The second generation of prototype boards came in a batch of 10 or 20 and had most hardware bugs from the first generation fixed. So, no more SMD stonehenge. Unlike the first generation, most of them were not mounted on scrap wood, but into prototype cases, with all peripherals. This was the first time we could do actual tests with the entire hardware, not just the electrical part. We gave some of those prototype devices to the client, so they could also do some initial testing.

        Four of those quite expensive prototype devices were also used in a transport test: Put the device in its transport box, put the box in a cardboard box, and drop that from a significant height onto a concrete floor, several times per side. No mercy. The client did all of those tests in parallel with four devices, and thereby destroyed hardware costing several k€. Lesson learned, the hard way: A simple cardboard box is not sufficient, it needs some padding, and the transport box also needed some more internal padding. We would have tested with only one device at a time, but the client insisted on fast progress. Well ...

        After that, we also dropped some devices, with better padding, damaged them significantly less, and learned how to improve the mechanics so it could survice the test without breaking apart. We repaired what could be repaired, then dropped the devices again. Finally we sorted the bits and pieces, including some remaining prototype case parts, into spare parts, parts for internal testing, scrap for ripping off spare parts, and junk. We build some test devices from the second quality, and still use these devices, full of scares, scratches, glue, and discolored plastic, as test devices for development. They have a few extra holes here and there, to access parts of the circuit not available on a normal device. Wires and, in one case, tubes dangle out, with switches, connectors, and extra electronics fixed to the case with velcro tape, Some other parts that serve no purpose for us, like a sticker with the client's support hotline, a carrying handle or a cover for power supply and service connectors, are usually removed from those devices, or were never mounted. Their serial numbers, usually hidden on a small sticker on the button, are written with a black marker on the front, so we can easily identify them. No customer of our client will ever see or touch those devices, and if they would see them, they would probably run away in terror. ;-)

        Two of the second quality boards that survived the drop tests replaced the first prototype boards on the scrap wood. We changed some parts on the boards to fine-tune the electronics, but after that, those second generation prototypes were identical to the first series production boards.

        One of the scrap wood systems was extended by a bunch of switches, wires, modified peripherals, banana jumpers for current and voltage measurement, and promoted to "test rig". With that setup, it was (and still is) possible to simulate almost all hardware problems, even the exotic ones: Broken wires, dead or broken peripherals, interrupted communication, and some more that I can't detail here.

        While working with the setup, when the first few serial boards have already been mounted into serial devices, we found out that we would need some more wires patched on the board to simulate more stuff. And one of those wires touched the +24V line.

        So yes, I killed a prototype board, but it was a second-generation board modified to be extremely similar to serial production boards. That's also the reason for the quite low damage: It was - except for minor details - electrically and mechanically identical to a serial production board, and it was already produced in a small series.

        After all, killing the board was kind of luck, because that allowed us to "steal" a brand new second series board with a better controller and some electrical improvements, so we could test the new stuff with the new software. We kept the other scrap wood system with a first series board for compatibility tests. Actually, that's currently my main target for development. If the software works on a series one board, it will also work with a series two board. So I test on a series one board. The software still has to support the series one boards, because several customers of our client have bought devices from the first series. Until all of those "old" devices have been exchanged for new devices or have ther main board swapped for a series two board, we can not release software that runs only on series two boards.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re^2: I came, I saw, I ...
by perldigious (Priest) on Jul 03, 2017 at 20:55 UTC

    Ha, don't worry, anybody who works with such things that says they have never done something like that is almost certainly lying. We have a "wall of flame" in one of the cubicles where I work to hang such things. My favorite addition was when someone we were helping design some boards for an Electric Vehicle somehow accidentally wired the full battery stack voltage directly to the microcontroller on one of the early prototype boards. There was literally a more or less perfect square hole the size of the IC burned through the board (along with some other components that exploded fantastically). I'm sure he got a long laugh out of the incident after he cleaned the mess out of his pants, he did send us the board back as a trophy after all. :-)

    I've accidentally welded a component to a circuit board once (and subsequently tripped one of the main breakers in the lab to make the other engineers very angry about interrupting their running tests) when I was working with a mixed high-voltage AC and low voltage DC board. I never did figure that one out. I was isolated through a variac, and the variac box was right next to me when it happened, and it let out a frighteningly loud 60Hz buzzing noise that nearly made me mess myself... Ah, good times, but I'd prefer to never go near anything higher voltage than a non-electric car battery ever again.

    Just another Perl hooker - But the purist monks keep telling me I should do it for love, not money.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1193999]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others learning in the Monastery: (3)
As of 2024-03-28 16:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found