Deterministic Wallets, Their Advantages and their Understated Flaws

cIf you have bot following the last year of progress te Bitcoin wallet development, you will have likely already heard of one of the latest trends te backend designbe: deterministic wallets. Unlike old-style Bitcoin wallets, which generate fresh Bitcoin addresses and private keys randomly spil needed, ter a deterministic wallet all of the gegevens is generated using a specific algorithm from a single seed. That is to say, if you write down the seed to your deterministic wallet, and then after six months your hard drive gets corrupted and the wallet unrecoverable, you can simply create a fresh wallet using the same seed and all of the addresses and private keys from your old wallet will come back again exactly spil they were before. This trend ter wallet development has received near-universal praise, and almost every Bitcoin client that intends to treat numerous addresses either already has a deterministic wallet implemented or is programma to create one.

However, deterministic wallets ter Bitcoin do not zekering there. Te fact, the latest deterministic wallets go beyond the plain vormgeving described above and have two key properties that are powerfully advertised by their developers. The very first of thesis properties is the concept of a “master public key“. A master public key is a key that can be generated from the wallet’s master private key (either the same thing spil the “seed” or a derivative of it) that has the power to generate all of the addresses ter a Bitcoin wallet, but none of the private keys. Thus, someone with access to a master public key can look at the balance of a deterministic wallet, but cannot actually spend the balance because they have no way of generating the private key corresponding to each address. The 2nd property is hierarchy: the private keys that you generate from a master private key are themselves master private keys and can te turn be treated spil deterministic wallets te their own right.

How do Deterministic Wallets Work?

Spil it turns out, there are two major types of deterministic wallets presently te use: Electrum wallets and BIP32 wallets, they use a very similar algorithm, permitting them both to have the master public key property, albeit the BIP32 wallets go further by also including the hierarchy property – Electrum wallets are designed to only go down one level, albeit one certainly could extend the Electrum protocol to make it hierarchical spil well.

The master public key property is perhaps the more surprising feature of deterministic wallets, and will be explored te detail very first. The reason why it works is that Bitcoin public keys – not fairly the same thing spil Bitcoin addresses but a closely related form – can be added and subtracted just like normal integers can (albeit, notably, you cannot multiply two public keys together), and thus the same arithmetic operations can be done on two “levels” – to generate private keys, the arithmetic is done on the level of integers, and to generate public keys it is done on the level of public keys.

The precise algorithm used by all deterministic wallet systems is this. To calculate the private key at index i (say i = Five), calculate an “offset” parameter using a function (technically, a hash) of the index and the master public key. Then, simply add the master private key and the offset together. To calculate the public key at index i, calculate the offset ter the same way, convert the offset to a public key, and add the master public key and the offset public key together.

Here are a few examples using Electrum wallets, done with my own pybitcointools library. Very first, wij generate a master private key and master public key from the seed:

Now, wij generate private key index zero:

Now, the public key:

And, just to voorstelling you that the math checks out:

Wij can repeat this with index 1, index Two, etc, you can attempt it yourself with your own Electrum wallet if you have one. The takeaway is this: you can securely give waterput your master public key ter an insecure place, or even give it out to third parties like auditors, if it makes life more convenient for you, just keep the master private key (and the seed) to yourself.

Hierarchy

Now, on to the hierarchical wallet property. This one is, once again, best described by simply showcasing it ter act:

The main use case for which this feature is advertised is ter hierarchical organizations: the treasurer of a company might have control overheen the root private key of a BIP0032 wallet, and then forearm off a “child” seed to each of the company’s departments who will then use that seed to operate their own wallet. The treasurer will have the master key to everything, but each department will only have the key to their own part of the funds.

And, of course, BIP32 has that same master public key property spil Electrum, but even stronger:

Thus, a BIP32 master private key can be thought of being at the top of an infinitely descending tree, capable of recovering every private key below it. And a BIP32 master public key is just the same, except it can only recover public keys and addresses. Another metaphorical way to think about it is te terms of the private keys sitting at the canopy level of a rainforest, and the public keys on the ground below them. You can navigate the same path on the private key and public key level, and no matter where you leap down your destination will be the same, but once you’re on the ground you can’t get back up (tree-climbing monkeys that can go up from the public key ground level to the private key canopy are, at least for now, purely theoretical).

An Understated Problem

From the descriptions wij eyed above, you likely understand that deterministic wallets have two properties. Very first, you can go from a parent key to a child key, but not ter switch sides. 2nd, you can give out your master public key with no risk to your funds – only your privacy. And this is how almost all people, at least those technically skilled enough to know what a deterministic wallet is, view BIP0032 wallets today. The specimen of a company, which forearms out child private keys to departments and master public keys to accountants and auditors, has come to take a central place te the mythology around the promise that BIP0032 wallets potentially hold. However, spil wij will see below, this description of hierarchical wallets is fatally flawed.

The problem is this: albeit you certainly can securely arm out child keys with no risk to the parent key, and you can palm out master public keys with no risk to the master private key, you cannot do both at the same time. The exploit for when that situation does arise is actually fairly ordinary, and can be done with two lines of pybitcointools code. I will use Electrum ter this example, since Electrum wallets are more semitransparent. Thesis are the same master public key and child private key I created above:

Spil wij eyed above, the very first private key is calculated by a formula which can be summarized spil mprivkey + calc_offset(mpubkey,index) . So, what do wij do? If you look for it closely, the response is remarkably visible:

And tada, wij get the master private key back. Now, wij can go ahead and pilfer all of the other addresses ter the wallet, even those which the wallet’s proprietor never intended to touch. I even included a instruction ter pybitcointools to make this more convenient for you:

BIP32 has the same vulnerability:

Te the interests of fairness, it is significant to note that this is not a unexpected fresh zero-day vulnerability discovery, many Bitcoin developers have known about this for a while. However, given the intuitive understanding of hierarchical deterministic wallets that many people implicitly promote, including the idea of handing out child private keys to organization departments and master public keys to auditors, this has a good potential to result te a security breach. It may only be a matter of time until a large organization determines to actually adopt a hierarchical deterministic wallet to protect its Bitcoin funds, and abruptly finds a collusion of one of its department goes and an auditor running off with the entire company funds. So the evident question is: can this be immovable? The reaction seems to be no, because the only operations that can be done with public keys is adding and subtracting them, the only way to implement a deterministic wallet with the master public key property is using the “offset” mechanism described here. If this is indeed true, then raising awareness is the only solution, together with a switch te BIP32 representation and te clients to make it clear that master public keys and hierarchical wallets do not mix.

There is one clever way te which this might be bypassed: making three hierarchical BIP32 wallets, with every address being a 2-of-3 multisignature address inbetween the three wallets down some particular child key derivation path. Then, an auditor can have one of the three master public keys, and search the blockchain for transactions whose script contains public keys generated from that master public key. The solution is ingewikkeld, not supported by any existing client, and far from flawless, but something like it seems to be the only way to get around the kwestie. Te most cases, however, simply not handing out the master public key may be the better treatment.

So what is the future of deterministic wallets? At this point, BIP0032 is arguably spil far spil wij can go, there are no known tricks te elliptic curve math that haven’t bot exploited yet. One demonstrable upgrade might be BIP0032 multisignature wallets, combining BIP0032’s hierarchical deterministic magic with an advanced feature te Bitcoin that permits you to send bitcoins to an address that requires two out of three given private keys to spend the funds. Another further direction is brainwallets. The two current competitors for memorizing a Bitcoin wallet are (1) choosing a password and using the password or a hash of the password spil a seed, and (Two) randomly generating a seed and converting the seed into a passphrase ter a way that can be reversed. The way that both approaches are implemented is presently somewhat flawed – the standard implementation of the very first treatment does far too little against brute force attacks, whereas the standard (Electrum) implementation of the 2nd treatment is too difficult to memorize – studies showcase that passphrases like “glow date cost bloody curve wheel cousin picture stadionring ultimately bubble press” are no lighter to memorize than random strings of characters of an equal security level, and they suggest no protection against leaving behind one or two words. Thesis are open problems – if you are a Bitcoin developer, you personally have the chance to come up with and standardize a solution.

Related movie: EVE Online – How To Gezond Magnate for Beginners ter Low-Sec Hacking Sites


Leave a Reply