Step 1 - Collecting Data

Open the Recipe for a Walk-through on How to Code

In this tutorial, we will go through the data collection phase and provide a guide to obtaining Token Metrics and market data from exchanges.

1.1 Preparation Before the Action

Here are the packages needed:

import requests
from pandas import json_normalize
from urllib.parse import urlencode

from typing import Dict, List, Union, Optional, Any
import warnings


Here is a valuable function built to help connect to our platform. Just enter your API key in <your API key> and explore the desired endpoints.

API_key = '<your API key>'

def tm_API(endpoint: str, payload: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
	"""Obtain data from the Token Metrics Data API

		endpoint (str): The endpoint of the API
		payload (Optional[Dict[str, Any]], optional): The parameters to send to the API. Defaults to None.

		Dict[str, Any]: The response from the API

	if payload:
		url = '' + endpoint + '?' + urlencode(payload)
		url = '' + endpoint 
	headers = {'Accept': 'application/json', 'Content-Type': 'application/json', 'api_key': API_key}
	response = requests.get(url, headers=headers)
	return response.json()

1.2 Obtain the Tokens Index from Tokens Endpoint

Tokens Index is a table with the Name, Symbol, and ID.

Once we have all our tools ready, what do we need first? It would be the Tokens !

As shown in our Guide, all the tokens or symbol information are stored in the Tokens endpoint. We can extract the token information with the following code:

endpoint = 'tokens'
params = {}
response = tm_API(endpoint,params)
coins = json_normalize(response['data'])
coins = coins.sort_values(by = 'TOKEN_ID').reset_index(drop = True)
coins[coins.NAME.isin(['Bitcoin','Ethereum','Litecoin'])].reset_index(drop = True)

We filtered the table with three coins: Bitcoin, Ethereum, and Litecoin for demo purposes.

To ensure that this tutorial presents the entire process in an easily understandable format from a professional trader's perspective, we will use a simple strategy rather than complex ones. So, before moving on to the following steps, we aim to provide a clear overview of the process. Long-Short Single asset trading strategy.

Here is an example of a simple Long-Short Single asset trading strategy:

  • We focus on Bitcoin only
  • There are only two positions : Long, Short

1.3 Obtain the Market Price from Exchange

import pandas as pd
import datetime as dt
import requests

Next, we need to collect the market quotes for Bitcoin.

# Obtain data from the Binance US API
params = {'symbol': 'BTCUSDT', 
       'interval': '1d', 
       'startTime': int(dt.datetime(2020,5,2).timestamp())*1000,
       'endTime': int(dt.datetime(2023,1,28).timestamp())*1000,
       'limit': 1000}

r = requests.get('', params = params).json()

# Convert to pandas dataframe
col = ['Open time','Open','High','Low','Close','Volume','Close time','Quote asset volume','Number of trades',
       'Taker buy volume','Taker buy quote asset volume','Ignore']
btcusdt = pd.DataFrame(r, columns = col)

btcusdt['Open time'] = btcusdt['Open time'].apply(lambda x: dt.datetime.fromtimestamp(x/1000)).dt.strftime('%Y-%m-%d')
btcusdt['Close time'] = btcusdt['Close time'].apply(lambda x: dt.datetime.fromtimestamp(x/1000)).dt.strftime('%Y-%m-%d')

# Clean up the data
btcusdt.rename(columns = {'Close time': 'Date'}, inplace = True)
btcusdt[['Open', 'High', 'Low', 'Close', 'Volume']] = btcusdt[['Open', 'High', 'Low', 'Close', 'Volume']].astype(float)
btcusdt = btcusdt[['Date','Open', 'High', 'Low', 'Close', 'Volume']]

We obtain the historical price of BTCUSDT from Binance US. After some data-cleaning steps, we get the following table:


And here is the plot of the data:

fig = plt.figure(figsize=(10,5))
sns.lineplot(x=pd.to_datetime(btcusdt['Date']), y=btcusdt['Close'])

1.4 Obtain the Data from Trader Grades endpoint

Our Trader Grades endpoint provides a comprehensive overview of price movements. This table can inspire many different trading strategies based on this endpoint.

endpoint = 'trader-grades'
params = {
	'tokens': '3375',
	'startDate': '2020-05-03T00:00:00.000Z',
	'endDate': '2023-01-27T00:00:00.000Z',
	'limit': 1000

r = tm_API(endpoint,params)
trader_grades = json_normalize(r['data'])

Once we extract the data from our API, we will only select the following features[['DATE','TA_GRADE','QUANT_GRADE','TM_TRADER_GRADE',]]. Next, we will do a simple transformation on the data.

# select the columns we want
btc_tg = trader_grades[['DATE','TA_GRADE','QUANT_GRADE','TM_TRADER_GRADE',]]

# merge the data
data = pd.merge(btc_tg, btcusdt, left_on='DATE', right_on='Date', how='left').drop('Date', axis=1)

# do some transformations
data['DayReturnPCT'] = (data.Close/data.Close.shift(-1) - 1)*100
data['ta_gradePCT'] = (data['TA_GRADE']/data['TA_GRADE'].shift(-1) - 1)*100
data['quant_gradePCT'] = (data['QUANT_GRADE']/data['QUANT_GRADE'].shift(-1) - 1)*100
data['tm_trader_gradePCT'] = (data['TM_TRADER_GRADE']/data['TM_TRADER_GRADE'].shift(-1) - 1)*100
data['DailyReturnPCT'] = (data.Close/data.Open - 1)*100

Now we have collected all the data we need. Please refer to the btc_tg.csv.

<class 'pandas.core.frame.DataFrame'>
Int64Index: 748 entries, 0 to 747
Data columns (total 14 columns):
 #   Column              Non-Null Count  Dtype         
---  ------              --------------  -----         
 0   DATE                748 non-null    datetime64[ns]
 1   TA_GRADE            747 non-null    float64       
 2   QUANT_GRADE         747 non-null    float64       
 3   TM_TRADER_GRADE     746 non-null    float64       
 4   Open                748 non-null    float64       
 5   High                748 non-null    float64       
 6   Low                 748 non-null    float64       
 7   Close               748 non-null    float64       
 8   Volume              748 non-null    float64       
 9   DayReturnPCT        747 non-null    float64       
 10  ta_gradePCT         745 non-null    float64       
 11  quant_gradePCT      745 non-null    float64       
 12  tm_trader_gradePCT  743 non-null    float64       
 13  DailyReturnPCT      748 non-null    float64       
dtypes: datetime64[ns](1), float64(13)
memory usage: 87.7 KB