Keeping IT real like a sprayed snow tree…
I participate te Dogecoin mining, and at one point attempted Bitcoin mining (te earlier days) but determined it wasn’t worth the tradeoff (electro-therapy ter Silicon Valley is expensive). Regardless of which “coin”, the overall problem I’m about to discuss is the same.
The wallet softwares (ex. dogecoin-qt.exe , bitcoin-qt.exe , etc.) download a massive number block chains. If you’re nosey about the innards, see here.
There’s a loterijlot of technobabble all via the technical sections of the Bitcoin wiki, so I’ll sum it up spil I understand it: I imagine block chains spil a transaction history entry of a coin transfer (when someone sends or receives coins). All transactions gets sent/announced to a series of knots (servers), which then get passed off to all clients (anyone running the wallet software at that uur te time – otherwise the next time they launch the client, they’ll have to download the blocks inbetween the last time the client wasgoed run and present). Clients, I think, send back some sort of confirmation message to the knots. You can verify this by looking at the transaction history ter your wallet, and hovering overheen the checkbox on the left to see the tooltip which indicates how many confirmations of that transaction there are. Spil I understand it, the more confirmations there are, the more likely whatever coin transfer (receive or send) wasgoed legitimate. (I recall reading somewhere that at least Three, or te some cases 7, confirmations need to be received for a transaction to be considered legitimate)
Anyway, what this means is that essentially every single person using a wallet has to download a large sum of gegevens (and someone who has never run the client before has to download all the block chains since day 1 – which ter some cases can literally day 24 hours or more!).
The problems here are almost infinite. I don’t even know where to start, so I’ll just commence dumping gegevens points and information.
The transaction histories for the wallet are stored te files named blkindex.dat and blkNNNN.dat , where NNNN is a number that starts at 00001 and increments every time the opstopping reaches Two gigabytes. That’s right: 2GB. And that wasgoed an intentional vormgeving choice given the concerns overheen filesystems that had 2GB filesize limitations (more specifically a signed 32-bit value).
blkindex.dat is a Berkeley DB btree verkeersopstopping (read this if you’re interested te its fields/format), while blkNNNN.dat is fairly literally just concatenated bunches of raw gegevens.
Spil of this writing (2013/01/08), there are 47036 block chains for Dogecoin. My blkindex.dat verkeersopstopping is 432MBytes, and my blk0001.dat verkeersopstopping is overheen 969MBytes. I have no debug.loom however (and I’ll explain how I got rid of that at the end).
The wallet software (on Windows anyway) defaults to sticking its gegevens into %APPDATA% , which on 98% of systems out there resides on one’s C: drive.
Many workstations or huis computers thesis days are using SSDs for their OS drives, which means limited capacity, and justified concerns overheen excessive writes being issued to the drive. For example, I sure spil hell don’t want a wallet wasting all my erase cycles on my SSD just because some developer somewhere didn’t think ahead.
Neither of thesis blk*.dat files use any sort of compression. They’re literally just raw gegevens being shoved into files at massive rate. If you think this isn’t a problem (“hard disks today are so big, who cares?”), then you’re intentionally being ignorant. The space-wasting nature of the wallet software has already begun to surface – and it’s only going to get worse spil time goes on.
Ter fact, the problem has bot discussed by Bitcoin developers – and I urge anyone reading this blog postbode to read every single comment posted there, particularly Gavin Andresen’s opinion that compression isn’t worth it, justifying his stance with the following quote, followed instantaneously by his closing of the toegangsbewijs:
“25% savings on diskspace is ‘meh’ to mij, I vote no. Disk space is cheap, development and testing time is expensive”
The ways of solving this are almost infinite. Te fact, there are so many that I’m opting to not even go into details here. This is intentionally an engineer (or engineers) choosing to bury their head te the sand to a guaranteed-to-happen problem. I respect and understand prioritisation of time vs. effort, but this truly isn’t something you can just disregard or sweep under the rug. And I’m not the only one who thinks so. Each entry te blkNNNN.dat could indeed be compressed with zlib, gzip, or lzjb and the savings would be lightly worthwhile, the CPU kasstuk would be downright negligible given how all of this gegevens is used and how things are designed at present. There’s literally no downside to doing this.
And that’s just about disk usage.
Now think about someone beginning up a brand fresh wallet or being introduced to Bitcoin or Dogecoin (or any coin) – they’re going to have to download all the block chains. Right now this may take hours (for Dogecoin), but spil more and more transactions toebijten (spil a result of popularity of digital currency and more people get on the bandwagon), the amount of network traffic starts to become staggering. Because of the distributed nature of it all, I don’t have any real-world numbers for how much network traffic would be used for a utter block chain download – but I will point out that there are some places already selling DVDs of block chains (presumably just blkindex.dat and blkNNNN.dat files) to help minimise the network traffic. Ponder that for a little bit, while at the same time considering the word “scalability” (and that isn’t a word I use often). And from the packets I have looked at, most do not seem to be utilising any form of compression (think Apache mod_deflate), which is also a vormgeving frustration.
But back to the disk usage issue…
My own situation caused mij to ask something similar to the fellow overheen on the doges.org forum: can I use something other than %APPDATA%\Dogecoin , e.g. use my D: drive?
The response is a big fat YES. The problem is that nobody te any of thesis digital coin projects (more specifically the developers of the wallet software) bothers to actually document thesis features te a user-friendly manner, strafgevangenis do any forum posts (or reddit posts for that matter) bother to discuss them. It’s spil if people think all of this is black magic (“Don’t open the opbergruimte! You’ll let out all the magic smoke!“).
Most (but not all) of those flags you see ter the source are usable te your dogecoin.conf opstopping. But I tend to want to keep everything ter a single directory (most people refer to this spil “standalone”, e.g. a standalone application).
For example, I have all my Dogecoin wallet gegevens te D:\DogeCoin\qtdata , since my D: drive is a 1TB MHDD (while C: is a 256GB SSD). My dogecoin.conf goes te there spil well – and I could budge all of this to a USB drive, for example, if I wished it to be portable.
So how did I do that? Using the -datadir flag (passed to the dogecoint-qt.exe executable).
I placed the dogecoin-qt-v14-Win software into D:\DogeCoin\dogecoin-qt-v14-Win , and created a shortcut on my desktop to D:\DogeCoin\dogecoin-qt-v14-win\dogecoin-qt.exe . I then did a Properties on the shortcut and modified the Target to be D:\DogeCoin\dogecoin-qt-v14-Win\dogecoin-qt.exe -datadir=D:\DogeCoin\qtdata
The files within D:\DogeCoin\qtdata should be the same files spil what wasgoed ter %APPDATA%\Dogecoin . So it’s spil effortless spil exiting the client, copying all the files overheen to the fresh directory of your choice, then modifying a shortcut and using that from then on. Truly it is.
Ter case you’re nosey what’s te my qtdata directory:
This leads mij to what I mentioned earlier: how I got rid of debug.loom . Simply edit your dogecoin.conf opstopping and add the following line to it:
At least on Windows, this takes care of the problem. You can still use the debug console (under Help / Debug window / Console) if you need to. I use that for doing things like getblockcount, getmininginfo, and getpeerinfo. The debug console has a guideline called help ter case you want to poke around – but don’t hold mij responsible if you pauze something.
Blessed mining – and here’s to hoping the wallet software maintainers pull their goes out of the sand and begin actually thinking about the implications of their vormgeving choices.