Collect transaction data

Ethereum transaction data

Ethereum is a global, decentralized platform for money and new kinds of applications. All data in Ethereum will be saved in 'block', such as the transactions among accounts, block timestamp, and so on.

In this section, we demonstrate how to collect transaction data on Ethereum by block order.

Start a crawl task

The following command will continuously collect transactions, from block number 19000000 to the latest block:

scrapy crawl trans.block.evm -a start_blk=19000000 -a providers=https://freerpc.merkle.io

You can find the transaction data on ./data, in which:

• BlockItem.csv: saves the metadata for blocks, such as minter, timestamp, and so on.

• TransactionItem.csv: saves the external transactions of blocks.

The output files are in CSV formats, just like this:

address_from,address_to,block_hash,block_number,gas,gas_price,input,nonce,timestamp,transaction_hash,transaction_index,value
0x8ef57238cd4178eed96cd30a13cbc7448925328e,0x0000001d0000f38cc10d0028474a9c180058b091,0xcf384012b91b081230cdf17a3f7dd370d8e67056058af6b272b3d54aa2714fac,19000000,297348,20065519804,0x52c0f5ad62af6f001072c9162d6532b903d894fccffd802b0c83027587949af8b82b56bbd5ceceb046cd4e8901911d283c0bbf6d1b60728040a8,5522,1705173443,0xc9ab90a1796613b0fbccb33827de583901d4c0775e89c0f25d604b7c3a71771e,0,64243894
0x75fe4611b9b91c22a6d37c264a449f245cb0d0a9,0xf3de3c0d654fda23dad170f0f320a92172509127,0xcf384012b91b081230cdf17a3f7dd370d8e67056058af6b272b3d54aa2714fac,19000000,307298,20565519804,0x9871efa4000000000000000000000000a6ac39ba1d41cbbe37136a657309c2862c20c9e2000000000000000000000000000000000000000000001aa467b055f8ed86be690000000000000000000000000000000000000000000000000f3ad745ea4127d90000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000230000000000000003b6d034028200dfc4a177b2ef62165edce6c9fb7bc1e997a30000000000000003b6d0340f5ad62af6f001072c9162d6532b903d894fccffd,1678,1705173443,0x83ceb937ad4533442a5e730718ab3c5757386bd2d472f18507db5ecfc7cd54b8,1,0
0x5effeee074527eb8fe28a90d6e57d634b068bbc8,0xb517850510997a34b4ddc8c3797b4f83fad510c4,0xcf384012b91b081230cdf17a3f7dd370d8e67056058af6b272b3d54aa2714fac,19000000,220718,20165519804,0x8fe965b300000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000065a2e8c000000000000000000000000000000000000000000000000000b1a2bc2ec500000000000000000000000000000000000000000000000000000608608defdd545800000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000001000000000000000000000000005effeee074527eb8fe28a90d6e57d634b068bbc80000000000000000000000000000000000000000000000000000000000000002000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc200000000000000000000000088f8ea35e55014faba05e562c98c4ee1cc4adc130000000000000000000000000000000000000000000000000000000000000001000000000000000000000000d2a52f45c74b358abe1428bc43f0ce9ddf130780,283,1705173443,0x72e16826b6b5c971c7a10b73d841f22a0a7650b11f12c392e3a991f35e3cf5a3,2,50000000000000000