PawelAn open-source, public, blockchain-based distributed ledger featuring smart contract functionality — Ethereum — no doubt today can be viewed as a game changer in both digital and financial world. It allows to develop applications based on new technology (blockchain) that can easily substitute many businesses where an interaction with 3rd parties is nearly impossible to be completely eliminated in real life. Smart contracts in Ethereum network are just a software code which vouchsafes to deliver the same product to anyone who runs the code. They can store the value and unlock it when specific conditions are met. All by software itself. At the right time. At the right place.
The scope of current decentralised applications (dApps) being developed making use of Ethereum architecture over the past few years is truly worth an admiration. Just to name a few key domains, we have Decentralised Finance (DeFi), Decentralized Exchanges (for the trading of crypto assets without the need for a trusted authority, e.g. the banks!), Oracles (protocols or services to connect Ethereum to offchain information), Gaming, Crypto Collectibles (items that can be collected and often used across many different games and dApps), Marketplaces, Identity (DApps that are building identity systems), Governance. All these form an Ethereum ecosystem which can be referred to as Built on Ethereum (ETH) blockchain.
Those of you who already know the topic can see lots of potential for future developments of dApps in any of above-mentioned fields. On the other hand you can learn, try, and start using many solutions already offered by ETH network. Let’s consider two blockchain-based businesses, dYdX and Hashmasks, to taste some flavours of Ethereum’s applications.
dYdX is a decentralised trading platform that currently supports margin trading, spot trading, lending, and borrowing. The platform runs on smart contracts on the Ethereum blockchain, and allows users to trade with no intermediaries! dYdX allows programmatic usage of dYdX in Python, too. Users of the protocol can trade with margin or buy a token (crypto-coin) with leverage. Pairs currently available are ETH-DAI, ETH-USDC, and DAI-USDC. Perpetuals trading with up to 10x leverage is offered for BTC-USD, ETH-USD, LINK-USD pairs.
Hashmasks represents crypto collectible business built on ETH. It is a Swiss living digital art collectible created by 70+ artists globally containing a collection of 16,000+ unique digital portraits. By holding the artwork, you accumulate the Name Changing Token (NCT) on a daily basis, which allows you to choose a name for your portrait on the Ethereum blockchain. The NCT serves only one single purpose: it allows its holder to give their Hashmask a unique name that is permanently stored and publicly visible on the network. Therefore, commoditising the name itself and making it the rarest of all attributes within the entire project. This opens up a whole new dimension for collectibles where the value hierarchy of the individual pieces of the whole collective art is highly impacted by the preferences of the consumers.
Regardless of the kind of a new business built on ETH, many of business-specific and associated coins (tokens) get a real-world traction, being priced in fiat currencies (USD, EUR). A freshly born initiative like Hashmasks’s NCT token is still not tradable at any crypto-exchange but worth U$0.011 at moment, experiencing +507% increase in value since its inception on Jan 28, 2021. On the other hand, the Yield Protocol‘s value (YIELD) dropped by -45.3% in trading at Gate.Io Bitcoin exchange platform since its first record on Mar 9, 2021.
In this post, using Python, we will show how to get an access to the most recent information on crypto-coins (valued by the market or already traded) related to new businesses built on ETH blockchain. This will help us to create a data pipeline for updates on new ventures and opportunities for investors. Within our analysis, we will focus on the coin profitability for tokens that appeared in the market since Jan 1, 2021 and show that 4 out of 17 grew by over 90% just in a couple of weeks. Given that framework, one can adopt all Python codes to individually crafted crypto-data mining and in-depth research.
1. Getting All Coin Data
Thanks to CryptoCompare’s API, we can directly fetch the most up-to-date data on all crypto-coins and tokens that this service has in its records. This will be our starting point:
# Built on ETH Blockchain: Calculating Profitability of
# New Emerging Crypto-Coins since their Inception
#
# (c) 2021 QuantAtRisk.com, by Pawel Lachowicz
import ccrypto as cc
import numpy as np
import pandas as pd
import requests
import json
from bs4 import BeautifulSoup
import pickle
import matplotlib.pyplot as plt
url = 'https://min-api.cryptocompare.com/data/all/coinlist?
&api_key=your_api_key_here'
# fetch the raw data for an URL address above
response = requests.get(url)
soup = BeautifulSoup(response.content, "html.parser")
dic = json.loads(soup.prettify()) # convert from json to dictionary
# save dictionoary in a file (binary format)
with open('dic.db', 'wb') as handle:
pickle.dump(dic, handle)
# read it back from the file
with open('dic.db', 'rb') as handle:
dic = pickle.load(handle)
Once you sign in to the service, you will have an ability to generate your own API key that allows for extended free downloads. It needs to be inserted in line #17. A handy guide to CryptoCompare’s API you will be able to read in my forthcoming book Cryptocurrencies with Python.
For reproducibility of the code I saved the data I was working with to the file (download: dic.db). Using code lines #28-30 the file can be read in.
We can find information on the total of:
data = dic['Data'] coins = sorted(list(data.keys())) print(len(coins))
6569 coins and tokens. The data structure of the dictionary is transparent and easy to figure out:
{'Response': 'Success',
'Message': 'Coin list succesfully returned!',
'Data': {'42': {'Id': '4321',
'Url': '/coins/42/overview',
'ImageUrl': '/media/35650717/42.jpg',
'ContentCreatedOn': 1427211129,
'Name': '42',
'Symbol': '42',
'CoinName': '42 Coin',
'FullName': '42 Coin (42)',
'Description': "Everything about 42 coin is 42 - apart from the transaction fees and difficulty
retargetting - 0.00000001 and 7.5mins. A scrypt coin with 42 coins max, a 42
second block time, with superblocks giving 10 times the standard block reward of
0.0000420 42's.42 coin is a cryptocurrency with completed emission, fair
distribution (no ICO, premine or instamine) and both private and public transaction
support. The maximum supply of 42 coins makes the remaining 41.99 extremely rare.
The innovative deflationary model provides a constant rise in incentives both for
miners and long term investors. 42-coin delivers a hybrid of Proof-of-Work and
Proof-of-Stake transaction confirmation methods and represents a new way of
securing the network against 51% attacks.",
'AssetTokenStatus': 'N/A',
'Algorithm': 'Scrypt',
'ProofType': 'PoW/PoS',
'SortOrder': '34',
'Sponsored': False,
'Taxonomy': {'Access': '',
'FCA': '',
'FINMA': '',
'Industry': '',
'CollateralizedAsset': '',
'CollateralizedAssetType': '',
'CollateralType': '',
'CollateralInfo': ''},
'Rating': {'Weiss': {'Rating': '',
'TechnologyAdoptionRating': '',
'MarketPerformanceRating': ''}},
'IsTrading': True,
'TotalCoinsMined': 0,
'BlockNumber': 0,
'NetHashesPerSecond': 0,
'BlockReward': 0,
'BlockTime': 0,
'AssetLaunchDate': '0000-00-00',
'MaxSupply': 0,
'MktCapPenalty': 0,
'IsUsedInDefi': 0,
'IsUsedInNft': 0},
'300': {'Id': '749869',
...
Not every single coin record has the same number of information (sub-keys). A good ideas idea is to create their complete list. You can do it making use of Python’s set as follows:
s = set()
for coin in coins:
tmp = list(data[coin].keys())
s.update(tmp)
keys = sorted(s)
print(keys)
what returns:
['Algorithm', 'AssetLaunchDate', * 'AssetTokenStatus', 'BlockNumber', 'BlockReward', 'BlockTime', 'BuiltOn', * 'CoinName', 'ContentCreatedOn', 'DecimalPoints', 'Description', * 'Difficulty', 'FullName', 'Id', 'ImageUrl', 'IsTrading', * 'IsUsedInDefi', 'IsUsedInNft', 'MaxSupply', 'MktCapPenalty', 'Name', 'NetHashesPerSecond', 'PlatformType', 'ProofType', 'Rating', 'SmartContractAddress', 'SortOrder', 'Sponsored', 'Symbol', 'Taxonomy', 'TotalCoinsMined', 'Url']
In our research we’re going to use only four keys marked above by the asterisks.
2. ‘Built On’ Statistics
Naturally, it is intriguing to check for your own curiosity all available values of the key BuiltOn. The best guess is that a large portion of all tokens have their roots in Ethereum network but are there any exceptions? With the below code we can generate basic statistics:
builton = list()
for coin in coins:
d = data[coin]
if d['IsTrading']:
try:
bo = d['BuiltOn']
check = True
except:
check = False
if check:
builton.extend(bo.split(','))
builton = sorted(builton)
# perform simple statistical analysis
from collections import Counter
ub = list(Counter(builton).keys())
nb = list(Counter(builton).values())
print('tBuilt-ontNo. of Coins')
print(36*'-')
for e in zip(ub, nb):
print('%16st%1g' %(e[0],e[1]))
We require two conditions to be met here: (a) the coin needs to active (line #48), and (b) the record BuildOn needs to exist (lines #49-53). If this is passed, the record BuiltOn can contain more than a single value. Therefore in line #55 we split a potential list into individual names and extend a global list. Finally, making use of collections library from the Python’s Standard Library, we generate a list of unique names found in builton list (variable ub) and the corresponding records (variable nb). The output of this code is as follows:
Build-on No. of Coins
------------------------------------
2017-10-25 1
ADA 1
ALGO 2
BNB 43
BTC 1
DOT 1
EOS 3
ETH 1133
FTM 1
GO 1
HT 1
IOST 1
KAVA 1
LUNA 2
Mainnet 4
NAS 2
NEO 10
NULS 1
OMG 1
OMNI 1
ONT 1
QTUM 1
RSK Network 1
SLP 1
SOL 1
STRAX 1
TRX 12
VET 3
WAN 1
WAVES 5
XLM 6
XMR 1
ZIL 1
luniverse 1
mainnet 3
This confirms that a huge number of coins were born in ETH world as we initially were guessing. But do not allow yourself to be stopped here.
The above outcome is a good research result opening new path for an investigation of other tokens created based on other/hybrid blockchain architectures.
For example, TRX is a MainNet token based on the TRON Protocol issued by TRON Foundation, known as TRONIX. TRX is the basic unit of accounts on the TRON blockchain. The value of all other tokens derives from that of TRON. TRX is also a natural medium currency for all TRC-based tokens. TRX connects the entire TRON ecosystem, with abundant application scenarios that power transactions and applications on the chain. According to CryptoCompare’s records, there are currently 12 tokens built on TRON blockchain and you can quickly get their names and descriptions by running:
# Example: Tokens built on TRX
for coin in coins:
d = data[coin]
if d['IsTrading']:
try:
bo = d['BuiltOn']
check = True
except:
check = False
if check:
if bo == 'TRX':
# display coin name
print(coin, end=' ')
# uncomment for full coin description
# print('n')
# print(d['Description'])
# print()
what returns:
BRG BTT JST KLV MNR OKS PEARL SUN USDJ WIN
3. Built On Ethereum Blockchain. Released in 2021.
Let’s reuse the code from the TRX example adopting it for ETH-based tokens. Also, let us refine the search among all available coins and check how many of them were released solely from the beginning of 2021. Our goal would be to find out if purchasing any one of them was a good/profitable investment decision (if the coin went into crypto-exchange for live trading). Otherwise, we can inspect the change in valuation of new tokens over the past 4.5 months, hence the underlying business.
We will use a help of supporting function which allows to calculate number of days between two dates:
from datetime import datetime
def time_between(d1, d2):
d1 = datetime.strptime(d1, "%Y-%m-%d")
d2 = datetime.strptime(d2, "%Y-%m-%d")
diff = abs((d2 - d1).days)
return diff/365.25, diff/(365.25/12), diff # years, months, days
Next, we will scan all coins built on ETH, no older than 2021-01-01. Here, a useful information we can fetch from the AssetLaunchDate record of the coin (line #116) concurrently checking the lifetime of the coin and saving information on the oldest one in line #127:
today = datetime.today().strftime('%Y-%m-%d')
timeseries = {}
ecoins = []
oldest, oldestdays = '', 0
for coin in coins:
d = data[coin]
if d['IsTrading']:
try:
bo = d['BuiltOn']
check = True
except:
check = False
if check:
if bo == 'ETH':
try:
check2 = True
diff = time_between(d['AssetLaunchDate'], today)
except:
check2 = False
if check2:
if diff[2] <= time_between('2021-01-01', today)[2]:
try:
fd = cc.getCryptoSeries(coin, 'USD', freq='d')
if fd.shape[0] > 0:
timeseries[coin] = fd
ecoins.append(coin)
if diff[2] > oldestdays:
oldest, oldestdays = coin, diff[2]
print('%10st%st%10.2f' % (coin, d['AssetLaunchDate'],
diff[2]), end=' ')
print(' ...' + coin + '/USD timeseries downloaded')
except:
pass
For the coins that meet our filtering, we also fetch and store close-price time series vs USD (if available in the records of CryptoCompare database). Some may correspond to USDT rather than USD directly. Keep that in mind.
The output I’ve got was:
AUCTION 2021-02-01 106.00 ...AUCTION/USD timeseries downloaded
BCUG 2021-03-15 64.00 ...BCUG/USD timeseries downloaded
BDP 2021-03-04 75.00 ...BDP/USD timeseries downloaded
BYN 2021-04-13 35.00 ...BYN/USD timeseries downloaded
CGG 2021-03-05 74.00 ...CGG/USD timeseries downloaded
COVER 2021-01-05 133.00 ...COVER/USD timeseries downloaded
DAO 2021-01-07 131.00 ...DAO/USD timeseries downloaded
DIGG 2021-01-22 116.00 ...DIGG/USD timeseries downloaded
FORTH 2021-04-20 28.00 ...FORTH/USD timeseries downloaded
GUM 2021-01-30 108.00 ...GUM/USD timeseries downloaded
INSUR 2021-03-04 75.00 ...INSUR/USD timeseries downloaded
KYL 2021-02-28 79.00 ...KYL/USD timeseries downloaded
LKR 2021-04-04 44.00 ...LKR/USD timeseries downloaded
NCT 2021-01-28 110.00 ...NCT/USD timeseries downloaded
PAINT 2021-01-22 116.00 ...PAINT/USD timeseries downloaded
SX 2021-01-15 123.00 ...SX/USD timeseries downloaded
YIELD 2021-03-04 75.00 ...YIELD/USD timeseries downloaded
I allowed to save the whole dictionary object, timeseries, in a file (timeseries.db). You can skip running lines #98-132 (comment them) and read the data directly from the file:
with open('timeseries.db', 'rb') as handle:
timeseries = pickle.load(handle)
knowing that, in the case of this dataset, there was:
oldest = 'COVER'
Moving forward, let’s rearrange a list of coins we will be working with, now to start from the ‘oldest’ one:
tsk = list(timeseries.keys()) tsk.remove(oldest) tsk = [oldest] + tsk
and merge them all into a single pandas’ DataFrame as follows:
fi = True
for coin in tsk:
if fi:
df = timeseries[coin]
fi = False
else:
df = pd.merge(left=df, right=timeseries[coin],
how='left', left_index=True, right_index=True)
A Python trick we do here is that within the 1st iteration of the loop (fi = True) the df‘s index is assigned/equal to COVER’s index. Since it is the longest one (i.e. it contains the largest number of dates/close-price data points), JOIN LEFT operation executed in line #149 matches any other (shorter) time-series’ indexes with this longest one. In consequence, you vouchsafe not to discard any data point from other time-series, arriving with complete DataFrame, df, in the end.
The estimation of coins performance (profitability or losses) over their lifetime can be simply done by grabbing the first price record $p0$ (not-a-NaN) and the most recent one pL available. That allows us to compare the overall PnL development in time:
bestpairs = list() # gather best performing crypto-pairs
print('%12s %12s %12s %11sn' % ('Pair','Start_Price','Last_Price','PnL'))
for pair in df.columns:
ts = df[pair].dropna()
p0, pL = ts[0], ts[-1] # first and last price (USD)
pnl = pL/p0 - 1 # profit and loss
print('%12s %12.3f %12.3f %10.1f%%' % (pair, p0, pL, pnl*100))
if pnl > 0:
coin = pair[:-3]
bestpairs.append([pair, 100*pnl])
where in Python list bestpairs we store these pairs with a positive profit development since inception (see: AssetLaunchDate key discussed earlier). For time being, we get:
Pair Start_Price Last_Price PnL
COVERUSD 447.000 416.130 -6.9%
AUCTIONUSD 21.450 40.870 90.5%
BCUGUSD 6.350 3.330 -47.6%
BDPUSD 0.002 0.001 -41.3%
BYNUSD 0.421 0.349 -16.9%
CGGUSD 1.827 1.071 -41.4%
DAOUSD 5.226 4.773 -8.7%
DIGGUSD 27022.220 50462.090 86.7%
FORTHUSD 35.990 28.930 -19.6%
GUMUSD 1.977 1.904 -3.7%
INSURUSD 6.389 5.146 -19.5%
KYLUSD 1.282 0.756 -41.0%
LKRUSD 0.004 0.004 -5.3%
NCTUSD 0.002 0.011 507.1%
PAINTUSD 0.001 0.001 -39.4%
SXUSD 0.122 0.684 461.6%
YIELDUSD 0.244 0.130 -46.5%
where
print(bestpairs)
looks like:
[['AUCTIONUSD', 90.53613053613051], ['DIGGUSD', 86.74294710057129], ['NCTUSD', 507.12234348075816], ['SXUSD', 461.623067228573]]
and after sorting of the Python list, being a collection of 2-element sub-lists, according to the sub-lists’ second element:
bestpairs.sort(key=lambda x: x[1], reverse=True) print(bestpairs)
we end up with a list of the best performers:
[['NCTUSD', 507.12234348075816], ['SXUSD', 461.623067228573], ['AUCTIONUSD', 90.53613053613051], ['DIGGUSD', 86.74294710057129]]
As you may find it, it’s a quite handy method to remember as it comes to sorting of the nested lists in a list. Worth taking a note and improve your Python.
4. Inspecting Performance of New Tokens
As a grande finale, we may visualise the cumulative PnL in a function of time for our best performing coins. In the code that follows, we use a raw close-price time-series for each token and derive on-the-fly the rate of return:
plt.figure(figsize=(12,6))
for pair, _ in bestpairs:
# get cum PnL vector
tmp = 100 * df[pair].dropna().diff().cumsum() / df[pair].dropna().iloc[0]
tmp.plot()
plt.title('Built on ETH Blockchain: Best Performing Coins vs USD released in 2021',
fontsize=14)
plt.legend(loc=2)
#plt.ylim([-50, 400])
plt.ylabel('Cummulative PnL since Inception', fontsize=12)
plt.grid()
For our four musketeers, it looks like:
Not every business is actively traded (e.g. the coin of NCT vs USD). It does not mean it is worthless. As we have mentioned in the beginning, the value grows and builds in time. The above four tokens are quite successful as we speak but the rest (now in a negative territory of ROI; see output after code line #162) may change their valuation. Investors sometimes risk their money and lose a lot but if they study a new business standing behind a newly released coin/token on the crypto-market, its price may increase from pennies to hundreds of dollars.
Opportunities are all around us. Every day. Choose wisely. Research, examine, ignore or invest.
