Hi, I'm looking for the data format if bi5 tick files after lzma decomp. I would like to process the data myself after having downloaded them. I have found no comprehensive documentation on that.
AFAIK it is:
int milliseconds since epoch
int ask
int bid
float askv
float bidv
Can anyone confirm that?
Is there any plain Java examples on that?
Thanks in advance
bi5 Data format
Re: bi5 Data format
Hi Michael,
This is the EURUSD tick, 30/12/2019, original file named 00h_ticks.bi5, decompressed (with 7-Zip) file name: 00h_ticks (42,400 bytes)
Text format (CSV)
20191230 00:00:00:218,1.11815,1.11812,1.12000000476837,0.75
The binary record size is 20 bytes long. So the file contains 2,119 records = ticks
Binary (hex)
The time is expressed as milliseconds since the HOUR of the file.
The hour is included in the filename, so no need to include it into the binary time.
00 00 00 AD = 218 mill = HH:00:00.218
Just one more note about time.
Because time (excluded the HH) can be max 59:59.999 = 3599000 mills = 00 36 EA 98, the first byte of time will be always zero.
I wonder if it is actually used for something else...
Maybe it is there just for record padding.
The next 2 numbers are expressed as (straight 4 bytes) "integers", i mean like the window calculator shows them to you.
00 01 B4 C7 = 111815 = 1.11815 (divided by 100,000)
00 01 B4 C4 = 111812 = 1.11812 (divided by 100,000)
The next last 2 numbers are expressed as 32 bit floats (IEEE-754): 1 bit (sign) + 8 bits (exponent) + 23 bits (mantissa)
I used this useful converter to check them out: https://www.h-schmidt.net/FloatConverter/IEEE754.html
3F 8F 5C 29 = 0.01111111.00011110101110000101001 = 1.12 (error due to conversion: 0.00000000476837)
3F 40 00 00 = 0.01111110.10000000000000000000000 = 0.75 (no conversion error)
I think the above 2 numbers should represent millions.
Hope this helps.
Bye
jdaniele
This is the EURUSD tick, 30/12/2019, original file named 00h_ticks.bi5, decompressed (with 7-Zip) file name: 00h_ticks (42,400 bytes)
Text format (CSV)
20191230 00:00:00:218,1.11815,1.11812,1.12000000476837,0.75
The binary record size is 20 bytes long. So the file contains 2,119 records = ticks
Binary (hex)
Code: Select all
mills - ask - bid - ask vol - bid vol
00 00 00 DA - 00 01 B4 C7 - 00 01 B4 C4 - 3F 8F 5C 29 - 3F 40 00 00
The hour is included in the filename, so no need to include it into the binary time.
00 00 00 AD = 218 mill = HH:00:00.218
Just one more note about time.
Because time (excluded the HH) can be max 59:59.999 = 3599000 mills = 00 36 EA 98, the first byte of time will be always zero.
I wonder if it is actually used for something else...
Maybe it is there just for record padding.
The next 2 numbers are expressed as (straight 4 bytes) "integers", i mean like the window calculator shows them to you.
00 01 B4 C7 = 111815 = 1.11815 (divided by 100,000)
00 01 B4 C4 = 111812 = 1.11812 (divided by 100,000)
The next last 2 numbers are expressed as 32 bit floats (IEEE-754): 1 bit (sign) + 8 bits (exponent) + 23 bits (mantissa)
I used this useful converter to check them out: https://www.h-schmidt.net/FloatConverter/IEEE754.html
3F 8F 5C 29 = 0.01111111.00011110101110000101001 = 1.12 (error due to conversion: 0.00000000476837)
3F 40 00 00 = 0.01111110.10000000000000000000000 = 0.75 (no conversion error)
I think the above 2 numbers should represent millions.
Hope this helps.
Bye
jdaniele
Last edited by jdaniele on Fri Feb 19, 2021 1:33 am, edited 1 time in total.