Category Archives: Computers

Hacking the slot machine

Posted on by

Wired reports that a Russian group designed a brilliant slot machine cheat that they’ve used to bilk casinos out of millions of dollars. The article is sketchy on technical details, but there’s enough information there for me to speculate how it was done.

Understand, I don’t know anything about the internals of the software running on these machines, but I know enough about pseudorandom number generators, their use and misuse, to offer a plausible explanation of the vulnerability and how it’s exploited. I also know a few people in the industry. What I describe below is possible, and from my experience quite likely to have happened. Whether it’s exactly what happened, or if it’s even close to the mark, I have no way of knowing.

First, a little background.

In modern computer controlled slot machines (say, anything built in the last 30 years), the machine uses random numbers to determine the results of a spin. In concept, this is like rolling dice, but the number of possibilities is huge: on the order of about four billion. In theory, every one of those four billion outcomes is equally likely every time you roll the dice. That would be true in practice if the computer were using truly random numbers.

But computers are deterministic; they don’t do random. Instead, they use algorithms that simulate randomness. As a group, these algorithms are called pseudorandom number generators, or PRNGs. You can probably guess that PRNGs differ in how well they simulate true randomness. They also differ in ease of implementation, speed, and something called “period.” You see, a PRNG is just a mathematical way to define a deterministic, finite sequence. Given a starting state (called the seed), the PRNG will generate a finite set of values before it “wraps around” to the beginning and starts generating the same sequence all over again. Period is the number of values generated before wrapping. If you know what PRNG is being used and you know the initial state (seed), then you know the sequence of numbers that will be generated.

The machines in question were purchased on the secondary market sometime before 2009. It’s probably safe to say that the machines were manufactured sometime between 1995 and 2005. During that era, the machines were almost certainly running 32 bit processors, and likely were generating 32 bit random numbers using PRNGs that maintained 32 bits of state. That means that there are 2^32 (four billion and change) possible starting states, each of which has a maximum period of 2^32. Overall, there are 2^64 possible states for the machine to be in. That’s a huge number, but it’s possible to compute and store every possible sequence so that, if somebody gave you a few dozen random numbers in sequence, you could find out where they came from and predict the next number. It’d take a few days and a few terabytes of disk storage to pre-compute all that, but you’d only have to do it once.

It’s likely that the PRNG used in these machines is a linear congruential generator which, although potentially good if implemented well, is easy to reverse-engineer. That is, given a relatively small sequence of generated numbers, it’s possible to compute the seed value and predict the next values in the sequence. All this can be done without knowing exactly which specific LCG algorithm is being used.

The hackers did have the source code of the program (or they disassembled the ROM), but they didn’t have access to the raw numbers as they were generated. Instead, they had to deduce the random number based on the outcome of the spin. But again, that just takes a little (okay, more than just a little, but not too much) computation time to create a table that maps reel positions to the random sequence.

My understanding is that slot machines continually generate random numbers on a schedule, even when the machine is idle. Every few milliseconds, a new number is generated. If it’s not used, then it’s discarded and the next number is generated. So if you know where you are in the sequence at a given time, then you can predict the number that will be generated at any time in the future. Assuming, of course, that your clock is synchronized with the slot machine’s clock.

If you refer back to the article, you’ll see that the agent who was working the machine would record the results of several spins, then walk away and consult his phone for a while before coming back to play. That phone consultation was almost certainly uploading the recorded information to the central site, which would crunch the numbers to determine where the machine was in the random sequence. The system knows which numbers in the sequence correspond to high payouts, so it can tell the phone app when to expect them. The agent then goes back to the machine and watches his phone while hovering his finger over the spin button. When the phone says spin, he hits the button.

The system isn’t perfect. With perhaps up to 200 random numbers being generated every second, and human reaction time being somewhat variable, no player will hit the big payout every time. But he’s increased his odds tremendously. Imagine somebody throwing one gold coin into a bucket of a million other coins, and another gold coin into a bucket of 200 other coins. You’re given the choice to blindly choose from one of the two buckets. Which would you choose from?

That might all sound complicated, but it’s really pretty simple in concept. All they did was create a map of the possibilities and devise a way to locate themselves on the map. Once you know where you are on the map, then the rest is a simple matter of counting your steps. Creating the map and the location algorithm likely took some doing, but conceptually it’s very simple.

The above explanation is overly broad, I’ll admit, and I wave my hand over a number of technical details, but people at work with whom I’ve discussed this generally agree that this is, at least in broad strokes, how the hackers did it. Understand, I work at a company that develops slot machine games for mobile devices, and several of the programmers here used to work for companies that make real slot machines. They know how these machines work.

When I originally saw the article, I assumed that some programmer had made a mistake in coding or using the PRNG. But after thinking about it more critically, I believe that these machines are representative of the state of the art in that era (1995-2005). I don’t think there was a design or implementation failure here. The only failure would be that of not imagining that in a few years it would be possible for somebody who didn’t have a supercomputer to compute the current machine state in a few minutes and exploit that knowledge. This isn’t a story about incompetence on the part of the game programmers, but rather a story about the cleverness of the crooks who pulled it off. I can admire the technical prowess it took to achieve the hack while still condemning the act itself and the people who perpetrated it.

Digital computers in an analog world

Posted on by

One of the machines our software controls builds oligonucleotides by pushing various chemicals through a fine glass mesh. To do that, the machine dispenses the chemical into a common line and then pumps it to the destination. Logically, the machine looks like this:


                          |--------|             |----------|
      +-------+----+------|  PUMP  |--+-----+----|  Column  |----+
      |       |    |      |--------|  |     |    |----------|    |
    Neutral   A    B                  |     |                    |
                                      |     |                    |
                    +-----+-----+-----+     +--------------------+---- Waste
                    |     |     |
                  Argon   S     O

Flow is from left to right. The chemicals at the top are delivered by turning the pump. If we want to push 100 microliters (100 µl) of A through the column, we first open the Neutral valve and run the pump until the entire line through the column is filled with the Neutral solution. Then, the software closes the Neutral valve and switches the output to target waste. Then the A valve is opened and the pump is turned long enough to inject 100 µl into the line. The output valve is switched to target the column, valve A is closed, Neutral is opened, and the pump turns to push the chemical through the column.

It’s a little more complicated than what I describe, of course, but that’s the gist of it. Pump delivery is relatively easy because we know the length (more importantly, the volumes) of each tube segment, and we know very precisely how much liquid is moved with each revolution of the pump. If we want 100 µl and the pump moves 20 µl per revolution, then we run the pump for five revolutions. The pump is operated by a stepper motor, so we can vary the pump speed by controlling the rate at which we issue step commands to the motor. All we have to do is tell the pump controller, “do X revolutions in Y seconds.” The controller takes care of it and reports back when it’s finished. If we want to deliver 20 µl in three minutes, we tell the pump to do one revolution in 3,000 milliseconds. The motor controller figures out how many microsteps that is, and sends the motor pulses as required.

For reasons I won’t go into, some chemicals are better delivered by pushing with an inert gas (Argon, in this case). Each of the substances on the bottom is sourced from a bottle that’s under positive pressure. If the S valve is opened, liquid starts flowing, and continues to flow until the valve is closed. We know that at our standard operating pressure, the liquid will flow at a rate of about one microliter every seven milliseconds. So if we want to inject 20 µl of S, we open the S valve for 140 milliseconds. We can then close the S valve and open the Argon valve long enough to push the chemical through the column.

The problem comes when we want to vary the speed of Argon delivery. With the pump, all we have to do is slow the pump speed. But we can’t vary the bottle pressure so we have to toggle the Argon valve to simulate pumping. And that’s where things get kind of tricky.

Imagine, for example, that you want to deliver 20 µl of S smoothly over three seconds, rather than at the standard rate of 140 ms. If we could switch the valve on and off quickly enough, that wouldn’t be too difficult. We already know that we can pump 1/7 µl in a millisecond, so if we could toggle the valve every millisecond we could have 140 pulses of 1 ms each, with a 20 ms wait between each pulse. That would deliver in 2,940 ms.

Unfortunately, those valves don’t turn on and off instantly. It takes about 25 milliseconds for a valve to open after it’s energized. And it takes another 25 milliseconds for the valve to close completely after we cut power to it. So we can’t switch the valve faster than once every 50 milliseconds. And during at least part of that time the valve is partially open. So some liquid flows before the 25 ms is up. We can measure that, of course, and we could calibrate each valve if we wanted to. In practice, that calibration isn’t all that helpful. We can do almost as well by taking an average of all the valves in the system.

If you’re wondering how long 50 milliseconds is, a typical eye blink is between 300 and 400 milliseconds. So, whereas you can blink your eyes maybe three times per second, I can toggle these valves on and off 20 times per second.

The other problem is that the computer controlling the valve doesn’t have infinite resolution. Wait precision is typically about five milliseconds. If I want a seven millisecond wait, I’ll typically get something between 7 and 12 ms.

All those uncertainties mean that if I want to open the valve for what I think is 7 ms to dispense 1 µl, it takes 57 ms and I could end up with 2 µl flowing from the valve. This is what we end up calibrating. The tube volume from the farthest Argon-propelled valve to the waste is, say, 4,000 µl. We’ll clear the line (blow it out with Argon), and then start toggling the S valve in one-µl pulses until liquid flows from the waste. If it takes 2,500 pulses, then we know that one pulse dispenses an average of 1.6 µl. If it takes 5,000 pulses, we know that one pulse is, on average, 0.8 µl. That’s the number we’ll use in our calculations.

Interfacing with the real world presents a lot of challenges, but it’s incredibly rewarding. I worked on this machine’s software for several months before I got to see one of them in operation; all my work had been against a simulator. It was excited when I finally got to watch one of these machines work in response to user input, knowing that my software had converted the user’s commands into commands that make the machine switch valves and pump liquid through the system. Controlling hardware like that is the most rewarding type of programming I’ve ever done. I did it years ago, and I’m re-discovering just how much fun it can be.

Samsung Galaxy Tab

Posted on by

I’m sitting on the couch, typing this on my new Samsung Galaxy Tab 2–a 10 inch tablet computer. I have had this machine in my hot little hands for approximately 4 hours.

My first impressions are mostly favorable. Initial setup was no problem. I’m still trying to understand the logic (?) behind the home screen, though. Pressing the “home” button doesn’t always take me to my home screen. There are six different screens on which I can place icons and widgets, and pressing “home” will take me to the proper one most of the time. Infrequently, with no apparent reason, it will take me to one of the others–sometimes one of the blank pages. Odd.

It would have been nice to get a manual wih this thing. I found several on Samsung’s site, but haven’t yet figured out which is the right one.

My only real complaint at the moment is that the tablet keeps losing its WiFi connection. I thought it was because I’m in the other room, with a couple of walls between me and the router, but it was losing the connection when I was in the same dang room, earlier. I’ll find out tomorrow if it’s my connection, or if there’s something screwy with this tablet.

I also got a faux leather case wih a Bluetooth keyboad. Typing on this litle keyboard is better than trying to type on the tablet’s screen, but it’s going to take some practice. There is only one shift key, some of the keys (like the apostrophe/quote) are in odd places, the shift has a maddening tendency to remain engaged for a little longer than necessary, resulting in things like “HEllo.” And the keys are so dang close together! That said, I think I can learn to write on this thing.

Which is one of the reasons I bought it. I’ve long wanted the ability to blog and to write things when I’m away. A laptop works okay, but it’s kind of clunky to cart around. This tablet, with the keyboad and a good writing application, just might do the trick. And there are any number of reading applications available. I do a lot of online reading as it is. With this thing, I can download PDFs and other files, and read them at my leisure, wherever I am. Or so I think.

By the way, I’m writing this in the free WordPress application for Android. Seems to work well. Let’s see if it posts.

The robot story

Posted on by

Many years ago, some friends and I decided to build a robot. Dean was the primary hardware guy, and I was in charge of writing most of the software. The robot was a pretty ambitious undertaking that we never finished because most of the development team (four of us) moved away or had to spend our time on other more pressing things.

Early on in the development, Dean was wirewrapping the main logic board (Z-80 based) and I was writing the firmware on a CP/M system. By the time he had the board wirewrapped, I had the first cut of the base code finished. We burned the firmware into an EEPROM, did what bench testing we could, and then plugged the board into robot. After checking all the connections, including the RS-232 cable that we would use to control the robot during this test, we all stepped back and Dean turned on the power.

The robot’s motor came from an electric lawn mower, and the power source was a very high capacity lead-acid battery. We’d done some testing of the drive train earlier and learned that the thing was capable of some rather high speeds. Probably overkill for a little robot, but we figured we’d never tell it to go that fast. Little did we know . . .

Dean turned on the power and the robot took off across the shop like a rocket. 10 feet away from the bench, the RS-232 connector disengaged and the robot continued the rest of the way across the room and smacked into a wall. Where it stayed, its wheels spinning madly while the four of us rolled on the floor, laughing.

After picking ourselves up off the floor and turning off the robot, Dean turned to me and said, “Doesn’t your software turn off the motor at boot like I told you?”

Me: “I thought so!”

Sure enough, we looked at the code, which read:

start:
    ; turn off the drive motor
    xor a
    out (DriveMotor),a

Dean’s instructions to me at the time were, “You control the motor’s speed by outputting a value from 0 to 255 to the motor control port.” So I figured that 0 would be the lowest possible speed (i.e. stop), and 255 would be “shoot across the room like a rocket.”

Of course, Dean looked at that code and saw the problem immediately. If he explained to me why he’d designed the logic so that 255 was stop and 0 was “go like a bat out of hell,” I don’t remember the specifics. All I know is that I thought that doing things backwards made little sense.

People often look at me funny when I ask them today for clarification. A music player API, for example, might have a Volume property that’s documented as taking a number from 0 to 100. When I ask the author if 0 is mute or max, he rolls his eyes and says, “mute, of course.” Usually. But over the years I’ve run into a few backwards APIs similar to the robot’s drive motor, most often when dealing with custom hardware. I’ve learned the hard way that it pays to be sure, even if it does result in some funny looks from time to time.

Lost functionality

Posted on by

Have you ever noticed that quite often “new and improved” means lost functionality? Consider, for example, the difference between watching a movie on DVD and watching Hulu or Netflix. Overall, I like the Netflix experience. But sometimes I miss something and want to back up a few seconds. With the DVD remote, I could press a button and start viewing the video in reverse. It took just a moment to back up to where I wanted to be and then I could hit the Play button again. That’s not possible with any of the video players I see online.

With the browser-based players and even with Microsoft Media Player, I have to grab the slider and move it back. But it’s very difficult or impossible to move the slider back just 10 seconds. The procedure is something like this:

  1. Note the current video time, if it’s displayed.
  2. Grab the slider.
  3. Very carefully move the slider to the left, noting the displayed time.
  4. No matter how careful you are, you will go back too far.
  5. Move to the right.
  6. Repeat steps 4 and 5 in a binary search pattern until you get to the proper time index.
  7. Let go and hope you got it right.

Because the slider is scaled, the longer the video is, the harder it is to get this right, and you can’t get infinite precision. For example, if you’re watching a 2-hour movie and the slider is 1,000 pixels wide, then every pixel on the slider equates to 7.2 seconds.

With the slider, it’s easier to make big forward or backward jumps, but fine tuning is impossible. Especially when your input device is a wireless keyboard with a trackball, and you’re lying back on the couch enjoying the movie. I don’t understand is why none of the players I’ve seen have “fast forward” or “reverse” buttons. It seems like that’d be fairly easy to implement and then we could have the best of both worlds.

The smart phone I want

Posted on by

People are often surprised that I don’t have a smart phone. I carry the absolute cheapest (i.e. free) phone that I could get with our calling plan. And I upgrade it about every two years. For free. And I’m pretty happy with that arrangement. Not perfectly happy, but the promise of a smart phone isn’t yet compelling enough for me to spend several hundred dollars on a device and a hundred dollars or more every month for a calling and data plan.

Some of you may recall that I had one of the earliest smart phones. It was kind of nice to check my email when I was out of town. Other than that, I pretty much hated the phone. It was based on the old Palm OS. By default, input was with a stylus using their Graffiti handwriting recognition. I purchased some software that converted the bottom part of the display into a keyboard so that I could tap on the thing rather than trying to teach it how to read my scribbling. I could get maybe 20 words per minute on the thing.

But 20 WPM is about 1/4 my normal typing speed. Not only that, I had to look at the keyboard while I was typing. And notes couldn’t be very long. And on and on and on. As a simple list manager, the device was excellent. Anything more involved was beyond the phone’s capabilities.

I’ve seen mobile devices come and go since then. I had high hopes for Microsoft’s tablet back in 2002, but that didn’t go anywhere. Phones kept getting better. The iPhone and its Android clones are quite impressive. So are the iPad and its Android clones. But they all have what, to me, is a fatal flaw: they’re consumption devices. As production devices, they mostly suck.

Among other things, I’m a writer. I’m not a great writer, but I scribble a lot. I’m forever taking notes, jotting things down, sitting at the computer exploring some idea or other. My drives are full of notes, sketches, attempts at fiction, outlines for think pieces, rants, and all manner of other scribblings. When I’m at home, I’m as likely to be writing something as reading. And that doesn’t even count the writing I do for work: emails, computer programs, documentation, and articles.

So the first time somebody put an iPad in my hands, the first thing I tried to do was post a blog entry. So much for that idea. It’s possible, but painful. I can’t imagine trying to do that with an iPhone or similar.  I’ve come to the conclusion that nobody makes the mobile device I want. And I really don’t understand why. But then, people tend to think that our ideas are obviously good.

If I can find a smart phone with the following features, I’ll upgrade. So far, I haven’t seen anything that even comes close.

  1. The ability to connect to my home computer through USB, or share data via USB thumb drives.
  2. Network access to shared drives or NAS devices on my local network would be nice, but not essential.
  3. Ability to connect a full-sized keyboard. I’m liking those roll-up keyboards.
  4. Ability to connect a decent-sized video display. I really like the new rollable displays, although I suspect that those aren’t affordable yet. It should be reasonably easy to find a display unit that I could use for this.
  5. A camera with which I can take a picture and save it to my computer. Having to transmit pictures to “the cloud” is idiotic. Why can’t I transfer them directly to my computer?

I seriously doubt that Apple has or will make a device that meets these requirements. Is there an Android device that has those capabilities?  I’d buy one if I can get the peripheral devices. It would be the perfect mobile device: small enough and powerful enough for everyday carrying around, and flexible enough so that I could actually use it for productive work. That would be worth having.

Until I can use my mobile device to produce content, I’ll stick with my basic phone. I’ve yet to see a compelling reason to change.

I need a new computer

Posted on by

I’m looking to buy or build a new development machine. My current machine is a Dell 490 case with an Intel Core 2 Quad processor running at 2.4 GHz. 16 GB of RAM and two 750 GB hard drives. I have a second machine (Core 2 Quad at 2.0 GHz, 16 GB RAM, two 500 GB drives). Together, the systems make for a pretty reasonable development environment, but they’re starting to show their age. The machines can’t easily be upgraded with newer technologies (USB 3.0, SATA 3.0, etc.). Worse, those machines are noisy.

I should be able to replace both of those machines with a single machine, and not lose any required functionality. I figure if I can load it up with a fast processor (perhaps two quad-core processors), 64 GB of RAM, and two 2-terabyte drives, I’ll be in good shape. With two video cards and monitors, I can easily have two virtual machines running concurrently. With four monitors (a possibility), the system should rock!

My number one requirement is that the system be quiet. I realize that there will be some noise. That’s unavoidable. But these 490s are crazy loud. I don’t really notice how loud until I turn the machines off and am amazed by the quiet.

I’ve seen Jeff Atwood’s most recent Building a PC blog (from a year ago), and he has some good suggestions. His needs and mine often differ, though. In particular, he thinks that 24 GB of RAM is overkill. Some of the stuff I do will eat 24 GB with no trouble. I have programs that can make good use of 64 GB or more.

One other thing I’d really like is an SSD to hold my boot partition as well as a “scratchpad” area that I can use for large data-intensive tasks. A 128 GB SSD is quite reasonable: from $150 to $250. 512 GB will go double that, but might be worthwhile.

There are other things that’d be nice to have, but my basic requirements are:

  • Quiet case.
  • Quad core or possibly dual quad core (unless somebody has an 8-core processor).
  • Minimum 32 GB of RAM. Preferably 64 GB.
  • Space for four SATA drives, plus DVD.
  • Space for two dual video cards (total of 4 monitors).

I don’t know yet what that’s going to run me, but price isn’t all that important. I sit in front of these machines for 12 hours a day. If I can build one that saves me time and makes my work more enjoyable, then it’ll pay for itself pretty quickly.

Suggestions welcome.

Virtualizing

Posted on by

I’ve come to the point where I have to install some virtual machines on my computer in order to do my work. “No problem,” I thought, “I’ll just download Hyper-V and go.”

Dream on.

You see, I’m running Windows 7 Ultimate. Hyper-V only runs on Windows Server 2008. That’s crazy. I can’t imagine why Microsoft can’t release a version of Hyper-V for Windows 7.

I can either re-image my machine with Windows Server, or I can buy VMWare Workstation. Or I suppose I could re-image my machine with Ubuntu Linux, install VMWare Workstation, and do all of my Windows work in virtual machines. The real choice, though, is whether I want to spend the time re-imaging my machine. VMWare Workstation is only $200. Considering the time and trouble involved in re-imaging a machine, I’m thinking that VMWare is the way to go.

On a related note, I ran across Microsoft’s Windows XP Mode download. What a well done package. A quick download (well, if 500 megabytes is “quick”), a no-hassle install, and I have a 32-bit version of Windows XP running in a virtual machine, with full access to my Windows 7 drives. It’s great for running those few 16-bit applications that I’ve been too lazy to port or to find 32-bit versions of. Highly recommended.

That XP Mode package is a brilliant piece of work, but Virtual PC (the virtualizer that XP Mode runs on) is 32-bit only. I need 64-bit, which Hyper-V supports, but Hyper-V isn’t available for Windows 7.

It confuses me sometimes how Microsoft can be so customer focused in some ways, and totally clueless to their customers’ needs in others.

Confusing error message

Posted on by

In order to set the heart rate training zones on my Garmin Edge 500 bicycling computer, I have to set the zones on my Garmin Connect account and then send the zones to the device. Garmin Connect lets me set zones for the default, for running, and for bicycling. I only use the device for bicycling, so I ignored the other two and set the three heart rate zones that I use for bicycling. Then I clicked the “Send to device” link, and got this error message:

Your device can only accept 5 HR zones.

I found that odd. “Perhaps,” I thought, “it can only accept five total.” So I deleted the default and running HR zones. Again with the error message. It took me a while to realize that the message was saying that I had to have exactly five zones for each of the activities.

Forget for the moment the idiocy of creating software that can’t deal with fewer than the maximum number of zones. Forget also the idiocy of not fixing that in one of their many firmware updates. They could have at least provided a more explicit error message such as, “Your device requires you to enter exactly five HR zones.”

Why is it that hardware manufacturers have such a difficult time with software?

Shopping for a monitor

Posted on by

I’ve put together a desktop computer to use for development at home, and I need to get a new monitor for it. The old ViewSonic 15″ LCD that we paid $1,200 for 10 years ago is still in good shape, but its maximum resolution of 1280 x 1024 is not big enough to do serious work. I borrowed a monitor from the office, but at 1366 x 768, it’s not any better than the ViewSonic. Ideally, I’d like what I have at the office: a 24″ LCD with 1920 x 1280 resolution. But that, as it turns out, is very expensive.

You can get 1080p (1920 x 1080) monitors really cheap: about $100 for a 20″ unit. If you want a 23″ or 24″, you’ll pay $125 to $175, depending on brand and features. If you want HDMI input, you’ll pay another premium. Still, you can get a really nice 24″ 1080p monitor for under $200. You can, of course, spend as much as you want. A 60″ LCD TV makes a really nice monitor. It’ll cost you some serious cash, and it still won’t do better than 1920 x 1080.

If you want 1920 x 1200, you pay a huge premium. I think I saw one online for about $275, but in most cases you’ll have to shell out at least $300 for that extra 120 vertical pixels. That might not seem like much to some, but for a programmer or writer, 120 pixels works out to maybe 10 more lines of text. Vertical screen space is precious. Not $15 per line, though. Since I’m not willing to pay the premium for 1920 x 1200, I’m going with a 23″, 1080p model.

Graphics cards are kind of weird, too. In general, there are two types: the $20 card that will do video, and the $120 high performance card that you use for action games. As with anything, the top price is however much you want to spend. The middle ground (between $20 and $120) is mostly populated by fanless versions of the $20 cards. Trying to cut down on noise, and because I’m not much of a gamer, I’m going for the fanless cheap card. $50 isn’t exactly cheap, but that dang computer (a Dell 490 with a Core 2 Quad running at 2.5 GHz) is already too noisy. I’m not going to add more noise.