Adobe Flash can be used to run complex psychology experiments over the web. We examine the reliability of using Flash to measure reaction times (RTs), using a simple binary choice task, implemented both in Flash and using a baseline known to record RTs with millisecond accuracy. Twenty-four participants were tested in the laboratory using both implementations; they also completed the Flash version on their own computer. RTs from Flash on users own computers were ~20ms slower than those using Flash in the lab, which were in turn ~10ms slower than baseline. RT standard deviations were similar in all conditions, suggesting that although Flash may overestimate RTs slightly, it does not appear to add significant noise to the data recorded.