ThetaData

Appearance

Pagination ​

Pagination allows the Theta Terminal to split up a large response over multiple API calls. This ultimately increases application performance and consumes less memory.

Behavior ​

The next_page field in the JSON header object will be populated (non-null) if the amount of ticks in the response exceeds the HTTP_TICK_LIM as specified in the config file. We do not recommend changing HTTP_TICK_LIM to any value higher than 10000000.

Disable Pagination ​

You can disable pagination by setting a HTTP_TICK_LIM of 50_000_000. This isn't recommend unless you allocate a lot of memory to the terminal.

Be Careful! ​

You should process / call the next page url immediately after receivng a request that has non-null next_page field. Pages will be discarded if the amount of pages waiting to be called by the user is 2 times the HTTP_CONCURRENCY value and the page was generated over HTTP_PAGE_EXPIREms ago. You can change the HTTP_PAGE_EXPIRE config value to match the expected time to process a single http response and call the next page. You will get 477 errors if you try to call a page that was discarded or that does not exist yet.

Page ID ​

The page ID will get incremented each time a new page is needed. A single request might have multiple pages that do not have successive page IDs if other requests also require new page IDs.

Syntax: 127.0.0.1:25510/v2/page/page_id

Where is the next page when requesting CSV data? ​

The Next-Pagefield can be located in the http response header. An easy Google search on how to access http headers from a http request in your programming language of choice should provide adequate information.

Let's Try It ​

Inital request ​

Copy and paste the url into your browser while the Theta Terminal is running.

http://127.0.0.1:25510/v2/hist/option/quote?root=QQQ&start_date=20231004&end_date=20231004&strike=356000&exp=20231004&right=C

Response ​

Below is the header object in the response to this request. Notice the next_page field contains a url to the next page of data.

json
{
{
	"header": {
		"latency_ms": 638,
		"next_page": "http://127.0.0.1:25510/v2/page/1",
		"format": ["ms_of_day","bid_size","bid_exchange","bid","bid_condition","ask_size","ask_exchange","ask","ask_condition","date"]
 	},

Query the next page. ​

Copy and paste the value of the next_page field from the initial request into your browser to get the next "page" of data. Notice that the next page number might not always be 1, so it is important you use the exact page number provided in the initial (parent) request.

http://127.0.0.1:25510/v2/page/1

First Page ​

json
{
	"header": {
		"latency_ms": 0,
		"next_page": "http://127.0.0.1:25510/v2/page/2",
		"format": ["ms_of_day","bid_size","bid_exchange","bid","bid_condition","ask_size","ask_exchange","ask","ask_condition","date"]
 	},

And so on... ​

Querying http://127.0.0.1:25510/v2/page/2 (the next_page field from the last request we made) returns another page.

When to stop? ​

You must keep querying the last page until the last_page field is null like the output below.

json

{
	"header": {
		"latency_ms": 0,
		"next_page": "null",
		"format": ["ms_of_day","bid_size","bid_exchange","bid","bid_condition","ask_size","ask_exchange","ask","ask_condition","date"]
 	},