arweave-nft-uploader is a Python tool to improve the experience of uploading NFTs to the Arweave storage for use with the Metaplex Candy Machine.

Overview

arweave-nft-uploader

arweave-nft-uploader is a Python tool to improve the experience of uploading NFTs to the Arweave storage for use with the Metaplex Candy Machine.

The tool has an additional (optional) feature to manage complex NFTs with multiple asset files (e.g. a PNG and an SVG). See more about it in this section.

This tool is for experienced users. I decline any responsibility for unneeded expenses caused by the incorrect usage of this tool. Read this guide carefully before using it.

Prerequisites

Installing

To install this tool run:

pip install -U git+https://github.com/0xEnrico/arweave-nft-uploader.git

Creating the Metaplex Candy Machine

The Metaplex Candy Machine can be created in the standard way, with a few differences in the command line and the asset folder, since we will manage the upload ourselves:

  • Prepare a which should contain only the first asset: 0.json and 0.png. Do NOT put other assets in that directory.
  • Initialize the Candy Machine program with the addition of the -n switch that specifies the total number of NFTs that will be uploaded, with a command line like this (for other command line options please refer to the Metaplex Candy Machine documentation):
ts-node ~/metaplex-foundation/metaplex/js/packages/cli/src/candy-machine-cli.ts upload <single asset directory> -n <total number of NFTs> --keypair <Solana keypair file> --env <Solana cluster env name>

Uploading assets to Arweave

From the same folder where you run candy-machine-cli.ts, you can invoke this command to start the upload from the which should contain ALL assets to be uploaded:

arweave-nft -e <Solana env name> -k <Arweave wallet json file> <full assets dir>

e.g.:

arweave-nft -e mainnet-beta -k my_arweave_wallet.json /path/to/my/asset/dir

Invoke arweave-nft -h to get a full list of the available options:

usage: arweave-nft [-h] [-e ENV] [-k KEYPAIR] [-v] [-c CACHE_NAME] [--force-upload] [--assets-from-json] directory

positional arguments:
  directory             Directory containing images named from 0-n

optional arguments:
  -h, --help            show this help message and exit
  -e ENV, --env ENV     Solana cluster env name (default: "devnet")
  -k KEYPAIR, --keypair KEYPAIR
                        Arweave wallet location (default: "--keypair not provided")
  -v, --verbose         increase output verbosity
  -c CACHE_NAME, --cache-name CACHE_NAME
                        Cache file name (default: "temp")
  --force-upload        Force upload all assets, even the ones that have already been uploaded
  --assets-from-json    If this flag is specified, assets file names are read from properties.files.uri/type (e.g. for
                        uploading both png and svg), instead of the default pair NNN.json/NNN.png

In case you get an error like this at the end, it could be caused by upload errors or invalid asset structure in the json file:

WARNING There have been 6 upload errors. Please review them and retry the upload with the same command

Please review the preceeding errors, fix the json files content if needed, then run arweave-nft again with the same command line to retry.

At the end, you should get this message, and you can proceed to the next section:

INFO Upload complete! Now you can update the index with 'candy-machine-cli.ts upload' using the full assets directory (see documentation)

Rebuilding the Candy Machine index

After the upload has completed successfully, you need to rebuild the Candy Machine index.

This time you have to run candy-machine-cli.ts upload using the :

ts-node ~/metaplex-foundation/metaplex/js/packages/cli/src/candy-machine-cli.ts upload <full assets directory> -n <total number of NFTs> --keypair <Solana keypair file> --env <Solana cluster env name>

⚠️ When you run this command, the command prompt MUST be in the same folder where you ran it previously to create the Candy Machine. ⚠️

Failure to do this will incur in unneeded expenses as you will recreate the Candy Machine and start upload assets through the Candy Machine itself.

The only thing you need to change is the argument of the command from to .

If you have done everything correctly at this point, it will not re-upload.

You will see a very quick Processing file log:

Processing file: 0
Processing file: 50
Processing file: 100
...
Processing file: 850
Processing file: 900
Processing file: 950

Then a slower set of Writing indices lines:

Writing indices 0-9
Writing indices 10-19
Writing indices 20-29
...
Writing indices 970-979
Writing indices 980-989
Writing indices 990-999
Done. Successful = true.

At this point the process is complete, and you can run candy-machine-cli.ts verify:

ts-node ~/metaplex-foundation/metaplex/js/packages/cli/src/candy-machine-cli.ts verify --keypair <Solana keypair file> --env <Solana cluster env name>

If you have followed everything correctly, verify will succeed for all files, and your upload will be done!

Complex NFTs with multiple asset files

You can upload complex NFTs with multiple asset files using the --assets-from-json option.

Your json files must contain a files section like the one below, where you specify the asset files (in this example 0.png and 0.svg, which must be in the same directory as the json file) in the uri fields and the corresponding MIME type in the type fields:

"properties": {
    "files": [
        {
            "uri": "0.png",
            "type": "image/png"
        },
        {
            "uri": "0.svg",
            "type": "image/svg+xml"
        }
    ],
...

arweave-nft-uploader will manage the upload of all the referenced files and will create a correct json file on the Arweave blockchain with all the links.

Donations

If this project provides you a smoother experience while uploading your NFT assets to Arweave, I will appreciate a small donation to my Arweave wallet :)

bxQ7fygEV2meOp_z_3TZyy-VWbSCuzYRWnIE0FANQZo
Comments
  • Metadata is uploaded incorrectly

    Metadata is uploaded incorrectly

    There is a bug in arweave-python-client where data passed directly to a Transaction object is not uploaded correctly. It computes data_root from the Base64URL encoded data so the nodes do not recognize it and do not accept the original data.

    An example. data_root computed from the raw data is yiIbdak6AXJ17hGF5EJC_FpleCvd3pBe2c4jSZpRDVM whereas the tx was signed with XTcZfQOhQ0byq0NjAfJbYynVMyX6fkd4IBIfRR3S0IA - the value consistent with computing data_root from a Base64URL encoded blob.

    One quick workaround I have in mind is to always pass data to Transaction via a file handler (then it goes through a different code path which does not appear to have this bug), i.e., here do:

    fp = tempfile.NamedTemporaryFile()
    fp.write(json.dumps(asset_data))
    fp.seek(0)
    tx = Transaction(wallet, file_handler=fp, file_path=fp.name)
    
    opened by ldmberman 5
  • Solana expenses question

    Solana expenses question

    Hi Enrico,

    I’ve successfully managed to upload my assets using your tool, but I am confused about Sol expenses. When I upload 10000 images using metaplex upload command it costs me around 17 SOLs. I hope(d) that this tool can make it much cheaper, because I would pay for uploading 1 image by SOL, then the rest by AR, and that’s it. Sadly, however after I rebuild Candy Machine, it’s again 17 SOLs. Is it possible that I did something wrong or there is just no way how to avoid paying all those fees on Solana? I was very careful about being in the same folder all the time, actually it had to be metaplex/js/packages/cli, otherwise it wouldn’t work.

    opened by eroliko 2
  • Failing at step one

    Failing at step one

    getting WARNING: Ignoring invalid distribution -p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -2p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -1p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -0p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution - (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -2p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -1p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -0p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution - (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -2p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -1p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -0p (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution - (c:\python310\lib\site-packages)

    opened by lm166796 2
  • Candy machine skipping tokens for low mint price

    Candy machine skipping tokens for low mint price

    Hi Enrico,

    Thanks for providing the tool, and everything went ok, but it seems the candy machine skips some token when low mint price is specified - just wanted to ensure that it has nothing to do with using this tool, as it is just setting up the storage? I filed this bug also at metaplex, but no response so far: https://github.com/metaplex-foundation/metaplex/issues/936

    Please help sir.

    Thanks

    opened by shibaji 1
  • Support for Bundlr

    Support for Bundlr

    Hey,

    Without the use of Bundlr - transactions posted to Arweave can either 1 - drop or 2 - not seed. Both of which are critical issues for NFTs.

    As the Metaplex core team did - there would be benefit in integrating with Bundlr to support guaranteed data seeding and also native payment in SOL

    opened by joshbenaron 1
  • Created cache file with png 0 and json 0 but still get not init warning

    Created cache file with png 0 and json 0 but still get not init warning

    Hi Enrico, Thanks for sharing this!

    I see an error when first running this tool about funds and then re-running I get the init error, any ideas please? :)

    'Transfer: insufficient lamports 1352748523, need 16713695280', 'Program 11111111111111111111111111111111 failed: custom program error: 0x1

    Cache file .cache\mainnet-beta-temp is not initialized with a candy machine program

    opened by 1Crypt0 3
  • Error: failed to get info about account

    Error: failed to get info about account

    INFO Upload complete! Now you can update the index with 'candy-machine-cli.ts upload' using the full assets directory (see documentation) arweave all look right

    and i found this when i start the next steps

    ts-node src/candy-machine-cli.ts upload metafak -k /home/bom/.config/solana/metafak.json --env mainnet-beta
    Beginning the upload for 3636 (png+json) pairs
    started at: 1637300223041
    wallet public key: 9BNJRejrb1Rsd1We5nJsCTsA1htdb8StFNLhD8nUt21Z
    (node:375) UnhandledPromiseRejectionWarning: Error: failed to get info about account 63N3bBDCVVzdCaofQAnY5s4kcZodw1evomptDgNQ4feK: FetchError: request to https://api.mainnet-beta.solana.com/ failed, reason: connect ECONNREFUSED 128.0.113.154:443
       at Connection.getAccountInfo (/home/bom/metaplex/js/node_modules/@solana/web3.js/src/connection.ts:2404:13)
       at processTicksAndRejections (internal/process/task_queues.js:95:5)
       at async Function.fetchIdl (/home/bom/metaplex/js/node_modules/@project-serum/anchor/src/program/index.ts:309:25)
    (Use `node --trace-warnings ...` to show where the warning was created)
    (node:375) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
    (node:375) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
    
    opened by weck25 3
  • FetchError - Getting this error when trying to verify.

    FetchError - Getting this error when trying to verify.

    C:\Users----\metaplex-master\metaplex\js\node_modules\node-fetch\lib\index.js:1461 reject(new FetchError(request to ${request.url} failed, reason: ${err.message}, 'system', err)); ^ FetchError: request to https://b4by665argmyijbk34jiju3jeryos4efvs7myy3mg7lgaff6nh2q.arweave.net/DwOPe6CJmYQkKt8ShNNpJHDpcIWsvsxjbDfWYBS-afU failed, reason: connect ETIMEDOUT 13.226.14.70:443 at ClientRequest. (C:\Users----\metaplex-master\metaplex\js\node_modules\node-fetch\lib\index.js:1461:11) at ClientRequest.emit (node:events:390:28) at ClientRequest.emit (node:domain:475:12) at TLSSocket.socketErrorListener (node:_http_client:447:9) at TLSSocket.emit (node:events:390:28) at TLSSocket.emit (node:domain:475:12) at emitErrorNT (node:internal/streams/destroy:157:8) at emitErrorCloseNT (node:internal/streams/destroy:122:3) at processTicksAndRejections (node:internal/process/task_queues:83:21) { type: 'system', errno: 'ETIMEDOUT', code: 'ETIMEDOUT'

    When I was doing the initial uploading Visual studio crashed but when I started it up again it was uploading so I thought it was fine

    opened by ritorubool 1
  • Error Rebuilding

    Error Rebuilding

    Keep getting error reuploading below is a snippet from when the error began

    Processing file: 15500 Processing file: 15550 Writing indices 0-9 Writing indices 1000-1009 Writing indices 2000-2009 Writing indices 3000-3009 Writing indices 4000-4009 Writing indices 5000-5009 Writing indices 6000-6009 Writing indices 7000-7009 Transaction simulation failed: Error processing Instruction 0: Program failed to complete Program cndyAnrLdpjq1Ssp1z8xxDsB8dxe7u4HL5Nxi2K5WXZ invoke [1] Program log: libstd rust_begin_panic Program log: panicked at 'range end index 2651 out of range for slice of length 496', nft-candy-machine/src/lib.rs:318:18 Program cndyAnrLdpjq1Ssp1z8xxDsB8dxe7u4HL5Nxi2K5WXZ consumed 200000 of 200000 compute units Program failed to complete: BPF program panicked Program cndyAnrLdpjq1Ssp1z8xxDsB8dxe7u4HL5Nxi2K5WXZ failed: Program failed to complete Translating error SendTransactionError: failed to send transaction: Transaction simulation failed: Error processing Instruction 0: Program failed to complete at Connection.sendEncodedTransaction (C:\Users\Lm166\Documents\Potato\metaplex\js\node_modules@solana\web3.js\src\connection.ts:3553:13) at processTicksAndRejections (node:internal/process/task_queues:96:5) at async Connection.sendRawTransaction (C:\Users\Lm166\Documents\Potato\metaplex\js\node_modules@solana\web3.js\src\connection.ts:3513:20) at async Object.sendAndConfirmRawTransaction (C:\Users\Lm166\Documents\Potato\metaplex\js\node_modules@solana\web3.js\src\util\send-and-confirm-raw-transaction.ts:27:21) at async Provider.send (C:\Users\Lm166\Documents\Potato\metaplex\js\node_modules@project-serum\anchor\src\provider.ts:112:18) at async Object.rpc [as addConfigLines] (C:\Users\Lm166\Documents\Potato\metaplex\js\node_modules@project-serum\anchor\src\program\namespace\rpc.ts:19:23) { logs: [ 'Program cndyAnrLdpjq1Ssp1z8xxDsB8dxe7u4HL5Nxi2K5WXZ invoke [1]', 'Program log: libstd rust_begin_panic', "Program log: panicked at 'range end index 2651 out of range for slice of length 496', nft-candy-machine/src/lib.rs:318:18", 'Program cndyAnrLdpjq1Ssp1z8xxDsB8dxe7u4HL5Nxi2K5WXZ consumed 200000 of 200000 compute units', 'Program failed to complete: BPF program panicked', 'Program cndyAnrLdpjq1Ssp1z8xxDsB8dxe7u4HL5Nxi2K5WXZ failed: Program failed to complete' ]

    Side note: Big thanks to the developer @0xEnrico , I've sent some AR tokens to your wallet for your help and contribution to the community.

    opened by lm166796 2
Owner
0xEnrico
Old school developer in crypto space
0xEnrico
OpenSea Bulk Uploader And Trader 100000 NFTs (MAC WINDOWS ANDROID LINUX) Automatically and massively upload and sell your non-fungible tokens on OpenSea using Python Selenium

OpenSea Bulk Uploader And Trader 100000 NFTs (MAC WINDOWS ANDROID LINUX) Automatically and massively upload and sell your non-fungible tokens on OpenS

ERC-7211 3 Mar 24, 2022
Uploader-Bot - A Modified Telegram Url Uploader Bot With Mongodb, Zee5, Sonyliv Support and Many Other Yt-dlp Sites

???????????????? ?????????????????? ?? APP_ID API_HASH TG_BOT_TOKEN DATABASE_URL

null 11 Sep 10, 2022
Bulk NFT uploader to OpenSea!

Bulk NFT Uploader Description Simple easy peasy python script which logins to opensea account using metamask and bulk uploads NFT to your default coll

Lakshya Khera 25 May 23, 2022
Irenedao-nft-generator - Original scripts used to generate IreneDAO NFTs

IreneDAO NFT Generator Scripts to generate IreneDAO NFT. Make sure you have Pill

libevm 60 Oct 27, 2022
NFT Generator - A NFT Generator created using Python

NFT_Generator v1 An NFT Generator created using Python. This NFT Generation tool

null 3 Dec 2, 2022
Nft-maker - Create your own NFT!

nft-maker How to If you're going to use this program, change the pictures in the "images" folder. All images must be of the same resolution and size.

Georgii Arakelian 4 Mar 13, 2022
Simple-nft-tutorial - A simple tutorial on making nft/memecoins on algorand

nft/memecoin Tutorial on Algorand Let's make a simple NFT/memecoin on the Algora

null 2 Feb 5, 2022
NFT Generator: A modular NFT generator application

NFT Generator A simple passion project done with the role to learn a bit about h

null 2 Aug 30, 2022
A pdisk uploader bot written in Python

Pdisk Uploader Bot ?? Upload on Pdisk by Url, File and also by direct forward post from other channel... Features Post to Post Conversion Url Upload D

Paritosh Kumar 33 Oct 21, 2022
This is Pdisk Upload Bot made using Python with Pyrogram Framework. Its capable of uploading direct download link with thumbnail or without thumbnail & with Title Support.

Pdisk-Upload-Bot Introduction This Is PDisk Upload Bot Used To Upload Direct Link To Pdisk With Thumb Support Deploy Heroku Deploy Local Deploy pip in

HEIMAN PICTURES 32 Oct 21, 2022
Async ShareX uploader written in python

Async ShareX uploader written in python

Jacob 2 Jan 7, 2022
This Python script will automate the process of uploading your project to GitHub.

ProjectToGithub This Python script will help you to upload your project to Github without having to type in any commands !!! Quick Start guide First C

Imira Randeniya 1 Sep 11, 2022
A complete Python application to automatize the process of uploading files to Amazon S3

Upload files or folders (even with subfolders) to Amazon S3 in a totally automatized way taking advantage of: Amazon S3 Multipart Upload: The uploaded

Pol Alzina 1 Nov 20, 2021
Python script to Funge NFTs.

Python script to Funge NFTs. It scrapes OpenSea for a given list of NFT collections and downloads a certain number of NFTs from each collection or the entire collections.

null 3 Apr 28, 2022
An automated, headless YouTube Uploader

An automated, headless YouTube Uploader Authors: Christian C., Moritz M., Luca S. Related Projects: YouTube Watcher, Twitch Compilation Creator, Neura

null 127 Dec 23, 2022
A simple url uploader bot with permenent thumbnail support

URL-Uploader A simple url uploader bot with permenent thumbnail support Scrapped some code from @SpEcHIDe's AnyDLBot Repository Please fork this repos

Fayas Noushad 40 Nov 29, 2021
Pdisk Uploader Bot

pdisk-bot pdisk uploader telegram bot How To Use Configs TG_BOT_TOKEN - Get bot token from @BotFather API_ID - From my.telegram.org API_HASH - From my

lokaman chendekar 25 Oct 21, 2022
Mega.nz to GDrive uploader

Mega.nz to GDrive uploader With this telegram bot you can download files from mega.nz and upload those files or telegram uploaded files to GDrive. You

null 30 Nov 13, 2022