Skepticoin is a peer-to-peer digital currency that enables you to send money online

I launched skepticoin-mine.exe. I see it all the time, there is no anything about my hashrate and reward.

Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
NETWORK - 4 connected peers:
    18.218.172.21:2412 OUTGOING,
      23.88.99.67:2412 OUTGOING,
     3.144.210.13:2412 OUTGOING,
    83.162.197.24:2412 OUTGOING,
CHAIN - Height = 172987, Date/time = 2021-10-29T01:03:38


Waiting for fresh chain
NETWORK - 4 connected peers:
    18.218.172.21:2412 OUTGOING,
      23.88.99.67:2412 OUTGOING,
     3.144.210.13:2412 OUTGOING,
    83.162.197.24:2412 OUTGOING,
CHAIN - Height = 172988, Date/time = 2021-10-29T01:03:47


Waiting for fresh chain
Waiting for fresh chain
NETWORK - 4 connected peers:
    18.218.172.21:2412 OUTGOING,
      23.88.99.67:2412 OUTGOING,
     3.144.210.13:2412 OUTGOING,
    83.162.197.24:2412 OUTGOING,
CHAIN - Height = 172989, Date/time = 2021-10-29T01:04:03


Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
NETWORK - 4 connected peers:
    18.218.172.21:2412 OUTGOING,
      23.88.99.67:2412 OUTGOING,
     3.144.210.13:2412 OUTGOING,
    83.162.197.24:2412 OUTGOING,
CHAIN - Height = 172990, Date/time = 2021-10-29T01:05:55


Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
NETWORK - 4 connected peers:
    18.218.172.21:2412 OUTGOING,
      23.88.99.67:2412 OUTGOING,
     3.144.210.13:2412 OUTGOING,
    83.162.197.24:2412 OUTGOING,
CHAIN - Height = 172991, Date/time = 2021-10-29T01:06:44


Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
NETWORK - 4 connected peers:
    18.218.172.21:2412 OUTGOING,
      23.88.99.67:2412 OUTGOING,
     3.144.210.13:2412 OUTGOING,
    83.162.197.24:2412 OUTGOING,
CHAIN - Height = 172992, Date/time = 2021-10-29T01:08:43


Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain
NETWORK - 4 connected peers:
    18.218.172.21:2412 OUTGOING,
      23.88.99.67:2412 OUTGOING,
     3.144.210.13:2412 OUTGOING,
    83.162.197.24:2412 OUTGOING,
CHAIN - Height = 172993, Date/time = 2021-10-29T01:11:10


Waiting for fresh chain
Waiting for fresh chain
Waiting for fresh chain

what can i do?

After seeing some activity here yesterday I tried to start up the script again to see what was new. I ran:

pip install --upgrade pip
pip install --upgrade skepticoin
skepticoin-mine

I'm not super familiar with python so maybe I didn't even update properly?

Waiting for fresh chain (your chain is 10,236,836 seconds too old for my tastes)

When launching Skepticoin Client the first time, without a peers.json, the client downloads the initial peer list from https://pastebin.com/raw/CcfPX9mS

This is a vulnerability as we don't know who all control the contents of this URL and what would happen if they stop updating it, or even worse, the contents get maliciously updated due to a hack.

The Skepticoin Client should accept a list of trusted peers.json URLs which are maintained by the Skepticoin Community.

This script allows you to run multiple miners without having to worry about wallet collisions causing crashes and loss of coin if 2 instances mine a block at the same time. The number of instances can be passed via the -n argument or changed dynamically while running using the up and down arrow keys.

It does not work on Linux at the moment due to a socket address reuse error which unfortunately I don't know enough about Linux to fix.

I ran a profiler and found most of the time loading the chain was spent on disk and sha256 hash operations. With this PR, chain loads approximately 20x faster on a cold-started machine, 3x faster on a pre-warmed machine.

The "shot from the hip" missed its target. v0.1.15 is pretty broken.

I suggest merging the two below and doing an emergency release:

• #68 move peer_urls.txt to python code, without this new users can't connect to the network
• #70 prevent too many open files error on non-windows platforms, which breaks network related code badly

@sashimi-houdini @atramos

NETWORK Nr. of connected peers: 0

CHAIN Height 71482 Date/time 2021-05-16T16:54:39

Waiting for fresh chain Uncaught exception in LocalPeer.run() Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/skepticoin/networking/peer.py", line 860, in run File "/usr/local/lib/python3.9/site-packages/skepticoin/networking/peer.py", line 850, in handle_selector_events File "/usr/local/lib/python3.9/site-packages/skepticoin/networking/peer.py", line 761, in handle_incoming_connection File "/usr/local/Cellar/[email protected]/3.9.2_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 293, in accept OSError: [Errno 24] Too many open files

NETWORK Nr. of connected peers: 62

Observed behavior: all skepticoin-xxx commands start reporting very low numbers of peers. (for non-listening nodes: likely 0; listening nodes might actually still have some incoming connections that keep them in the network, I didn't research further)

Hypothesized cause: multiple processes writing to peers.json at the same time, corrupting the file.

Workaround: remove peers.json and restart your command (but don't multi-mine for now).

Proper solution(s) (preferably both in parallel in separate PRs):

• there should be only one process writing to peers.json at a time.
• properly research / deeply think about other touching of the file system, and how similar corruption could occur for other files too.
• add some robustness around the reading of peers.json... any failure to read or parse that file should probably lead to a loud and observable error, rather than the implicit peer-disconnect that we do now.

~Running on master, commit id 6cc1d55aef061cfcd3c53211307ba11eea5e93da:~ This issue probably affects all versions, past and present.

~Recently I submitted a PR which restored functionality on Windows by limiting the outbound sockets to 64. It appears we may have gone over some other limit, because now it has stopped working again.~ There's a problem with the initial peers.json file.

Here's a representative chunk of the output with --log-to-stdout:

Waiting for peers
2021-06-05 11:53:29,516 ConnectedRemotePeer.handle_can_send()
2021-06-05 11:53:29,516   138.68.170.71 Disconnecting remote peer [WinError 10057] A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
2021-06-05 11:53:29,520   138.68.170.71 LocalPeer.disconnect(OS error)
2021-06-05 11:53:29,529   138.68.170.71 NetworkManager.handle_peer_disconnected()
2021-06-05 11:53:29,536 ConnectedRemotePeer.handle_can_send()
2021-06-05 11:53:29,538   83.162.197.24 Disconnecting remote peer [WinError 10057] A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
2021-06-05 11:53:29,538   83.162.197.24 LocalPeer.disconnect(OS error)
2021-06-05 11:53:29,545   83.162.197.24 NetworkManager.handle_peer_disconnected()
2021-06-05 11:53:29,893 ConnectedRemotePeer.handle_can_send()
2021-06-05 11:53:29,893  60.243.226.191 Disconnecting remote peer [WinError 10057] A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
2021-06-05 11:53:29,896  60.243.226.191 LocalPeer.disconnect(OS error)
2021-06-05 11:53:29,897  60.243.226.191 NetworkManager.handle_peer_disconnected()
2021-06-05 11:53:29,912 ConnectedRemotePeer.handle_can_send()
2021-06-05 11:53:29,912     27.7.27.119 Disconnecting remote peer [WinError 10057] A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
2021-06-05 11:53:29,913     27.7.27.119 LocalPeer.disconnect(OS error)
2021-06-05 11:53:29,913     27.7.27.119 NetworkManager.handle_peer_disconnected()

Currently there are 7 open PR's ready to be reviewed and merged.

We are piling up, which is both good and bad. It's good to have people play around with the code, fix bugs and add features. And it's bad because it will take more time to resolve conflicts as more pending PR's are made.

@sashimi-houdini I suggest to add more people to get merge permissions, so we can move faster.

Obviously I volunteer myself. Maybe @atramos is also interested?

And no, I don't intend to merge my own work without others having looked at it. Maybe we can require at least one approval from another contributor? (Does github support that? I'm not too familiar with this platform.)

In any case, I think it's best to review/merge PR's in the order that they were submitted.

In this PR I want to:

• Have one NetworkingThread: in MinerWatcher
• Load the chain once: in MinerWatcher
• Handle the wallet once: in ... you guessed it

Effectively, Miner should be stripped to:

• calling scrypt
• facilitating the call to scrypt (including dealing with its return value)
• any other bookkeeping that cannot be done in MinerWatcher

At time of posting, this PR is ahead of my branches in #16 and #26, so those should be merged before this PR. Furthermore, the implementation is not done yet.

Recently saw this in my logs: Loading initial list of 508 peers

Of these 508, only 5 or 6 are valid peers.

Then I remembered local_peer.py has this code:

MAX_SELECTOR_SIZE_BY_PLATFORM: Dict[str, int] = {
    "win32": 64,
    "linux": 512,
}

I wondered I think some weird things will start to happen.

Then I saw this in the logs too, in a unusual spot after "block found", not where I normally see it near start-up: ValueError: Invalid file descriptor: -1 at remote_peer.py:256

For example, the fix in commit 0ae787ae5b4d199cda001119ee8bdd9428f12b76:

The parent commit throws an E275 in the github flake8 action.

Running locally as part of commit or with pre-commit run --all-files does not produce any error.

As the size of the Skepticoin blockchain has grown, the scalability bottlenecks have moved around to different parts of the code.

unspent_transaction_outs_by_hash (UTObH) is currently one of the largest contributors to slow start-up times.

Throughout the codebase, unspent_transaction_outs (UTO) is the set of Outputs minus the Inputs that used up those Outputs, aggregated at a given block in the chain, and indexed by Output key. UTObH is the collection of all such UTO maps at every block, indexed by block (i.e. UTO is virtually a Block attribute). Currently the UTObH is pre-computed at every height of every fork of the chain during add_block_no_validation, resulting in a slowdown of both the disk read and the initial block download.

An optimization opportunity exists because once the application is running, it actually doesn't care about all the UTOs that were pre-computed, or even all the O's in a given block's UTO. Only the Outputs that are currently being used as an Input in the next Transaction are important (or Outputs matching a Public Key, if computing balances). Some kind of Linked List might be a better solution.

I started thinking about the code for this but then my head wanted to explode, so I decided to write this ticket instead.

As seen in issue #110 , there is a need for some wallet management functions, such as some combination of the following:

• Create wallet with larger number of keys
• Merge wallets
• Add new keys to an existing wallet

In particular for downloading the chain "waiting for fresh chain" may not be clear to endusers.

I suggest using "Downloading latest blocks of chain". A download ETA would help as well.

This Draft PR is an initial proof of concept to demonstrate one possible architecture for a Skepticoin GUI.

Architecture:

The basic idea is to run an embedded HTTP Server in the Skepticoin Python code, and fire up a browser based app. The embedded HTTP Server has access to all the Skepticoin data structures such as wallet, blockchain, and peer connections. The browser based app uses a combination of GET api calls and EventSource to get information from the Python runtime.

Testing:

Run from this branch with:

python -c 'from skepticoin.scripts.gui import main; main()'