Other people have suggested that this black box should store public/private key pairs generated from the user's password for each user on the exchange. So when a user signs up for an account on the exchange, Javascript code generates a private key from the user's password, client side. The corresponding public key is sent and stored in the offline transaction signing box. Whenever the user wants to initiate a withdrawal, the transaction signing box creates a random number that needs to the signed with the private key that corresponds to the public key it has on store. This way an attacker need to compromise a server and install an eaves dropping application that replaces new users' (or existing users changing their password) real public keys with its own. Just breaking into the server wouldn't do the attacker any good at first.
Bitcoin supports "cold storage" which is effectively a wallet that can receive funds, but is offline so you cannot transfer funds out of it. That is, the private key is stored offline.
There is really no excuse for exchanges not using cold storage.
I think this will require too much manual intervention to be viable. Customers not being able to withdraw their funds because they've been sent to the cold wallet makes them unhappy with the service.
You could handle withdrawals out of a float fund, without actually reconciling against the user funds until an offline process completes. This way, at most, your float is at risk, and it's your money, not the customer's.
Other people have suggested that this black box should store public/private key pairs generated from the user's password for each user on the exchange. So when a user signs up for an account on the exchange, Javascript code generates a private key from the user's password, client side. The corresponding public key is sent and stored in the offline transaction signing box. Whenever the user wants to initiate a withdrawal, the transaction signing box creates a random number that needs to the signed with the private key that corresponds to the public key it has on store. This way an attacker need to compromise a server and install an eaves dropping application that replaces new users' (or existing users changing their password) real public keys with its own. Just breaking into the server wouldn't do the attacker any good at first.