Different results

[rpg]

Hi
I've started using tickstory today; but I only use it to export HST files into mt4 (not FXT).
Trying for an initial test I got a few problems. Here's are my steps:

1. Download AUDUSD from 1.1.2014 to 1.1.2015
2. Clear contents of MT4 history folder (also tester history), export HST files to MT4 under a demo FxPro.com account ( UTC +2 with European DST)
3. Run MACD sample EA. period M30 and modeling every tick using the exact dates mentioned in step 1.
4. Closing MT4 and repeating from step 2.

and here are the problems:

Even though "Include Weekends" was not selected during export, there are several occurrences of Sunday data.
Without any changes in data exporting settings or backtesting settings, each time I get a different backtesting result. Mismatched charts errors, modeling quality N/A, different number of trades and ...
Another weird thing is the FXT file which MT4 generates from the exported data. Sometimes its size is near 1 GB (which I think this should be normal), and sometimes it's just 680 kb!

As I said I do not change even smallest settings and options every time I repeat the process, but why different outcomes?
Thanks in advance

[tickstory]

Hi Rpg,

Please make sure that you are launching MT4 via Tickstory's "Launch MT4" button prior to backtesting. Failure to do so will result in the problems you're describing. If your results do not show 99.9% modelling quality then you have missed a step somewhere.

As for the Sunday data, Tickstory will filter all the data from weekends in the data's original timezone and then subsequently perform the time-shift. I imagine that this time-shift could be causing the data to move into the weekend.

Hope this helps.

[rpg]

Please make sure that you are launching MT4 via Tickstory's "Launch MT4" button prior to backtesting. Failure to do so will result in the problems you're describing. If your results do not show 99.9% modelling quality then you have missed a step somewhere.

Launching MT4 via tickstry works fine with 99.9% modeling quality. But what I wanted to do was just exporting HST files into MT4 and let metatrader build its own FXT files. I mean just using exported data like any other sources (similar to MT4's own history center data). Do I still need to launch MT4 thru tockstory even if I don't want real tick modeling?

As for the Sunday data, Tickstory will filter all the data from weekends in the data's original timezone and then subsequently perform the time-shift. I imagine that this time-shift could be causing the data to move into the weekend.

I checked it closely and I found there are four Sunday data appearances on my chart during 2014. Three in March and one in October.
And I think I've found why this happens. You probably have defined wrong DST switching dates for London (which are 30 March and 26 October in 2014), seems that you have used those dates of the United States (9 March, 2 November). That's why Sunday bars appear 3 weeks in March and once in the last week of October.

[tickstory]

Hi Rpg,

If you don't want to use tick (FXT) data that you do not need to launch via MT4. You will get the modelling quality n/a and potentially have mismatch errors (usually relating to volume since volume is set to 1 by default).

Regarding dates, try pressing the little 'i'/information button near the 'Timezone' box. This will display the time-adjustments that the system will automatically made for DST. This information is taken directly from the Windows Operating System.

Hope this helps.

[rpg]

Hi tickstory and thanks for your useful replies. I've sorted out mismatched charts errors and modeling quality issues; but still struggling with time shift and daylight saving time problems. I checked the "timezone info" for "UTC+2 with european/london DST":

timeinfo.png (8.7 KiB) Viewed 25658 times

I'm not sure what 1/Mar and 1/Oct mean (maybe 1st Sunday of the month?), but I still think those shifts that happen on my charts are based on U.S. DST shift, not London. I really need help to get rid of those Sunday bars


I have also a couple of more questions:
1. Number of ticks in Dukascopy data do not match to exported data. Please see attached below:

There are 29 ticks in selected minute, but in metatrader it show 67 ticks.

2. I searched on the forum for this one, but didn't get a clear answer. And the problem is why last 2 hours of Fridays are missed? Here I quote your own words:

The 'filter weekend' option will filter all tick data that occurs after Friday 22:00 hrs (UTC) and before Sunday 21:00hrs (UTC). It will then perform the time-shift to the selected time-zone and create bar data from this. This means that if the converted times fall between Monday 00:00-23:59hrs, then yes, everything will be seen on that bar.

Seems that "filter weekend" filters data after Friday 20:00? If not what could have caused the missing?

[rpg]

Okay, finally I figured out how to get rid of Sunday bars. EST +7 is the key (using UTC +2 with London DST results Sunday data in March and October); but couldn't find an answer for my last two questions:
1. Tick volumes: why they don't match the number of ticks in source data?
2. Why last two hours of Fridays are missed?

I gave CSV2FXT a try; apparently it generates exact tick volume and also full Friday bars.

[tickstory]

Hi Rpg,

Thanks for the more detailed information. Regarding your questions:

1) When exporting 'volume', the current version of Tickstory will export the actual bid/ask volume information. By default, however, the 'Suppress volume' option is checked which means that all volume will read '1'. There is further discussion on the topic of volume here: http://tickstory.com/forum/viewtopic.php?f=2&t=222

2) We are currently investigating further with the information you have given us and will get back to you.

Regards.

[tickstory]

Hi Rpg,

Attached is a screenshot from Dukascopy's own JForex platform showing the same day you mentioned. As you can see from the two screenshots, the last bar for 28th November 2014 is 21:45. The next session starts on 30th November at 22:00. We have also confirmed this from the raw historical data files that Dukascopy is delivering. You can also confirm yourself by going to the directory:

<StoreLocationFolder>\AUDUSD\2014\10\28 (the "10" represents November as the numbering is zero-based)

and noting that the 22hr and 23hr files are empty which means no trading is occurring at those times.

If you are getting additional data from other programs it is possible that it is being padded out by flat bars.

Hope this answers your questions.

[rpg]

Dear ticktory
Dukascopy's data timestamp is in GMT, and the broker which I export the data into, uses GMT +2. So if last bar of Dukascopy is 21:00 the exported bar should be 23:00. And those bars 22:00 and 23:00 in dukascopy data which are empty, are shifted into Saturday 00:00 and 01:00, when the market is closed, so they are empty.
Here I compare three different charts for 2014 November 28th. First chart shows the exported data using tickstory; second one shows CSV2TXT generated export and last one shows the broker's own data.

1. Tickstory

2. CSV2FXT (I used a CSV file generated by tickstory and then exported it into Metatrader using CSV2TXT)

3. FxPro's own data

As you can see, in all 3 charts all other bars are the same, except for last two bars of Friday, which tickstory misses to export.

[tickstory]

Hi Rpg,

Thanks for the further information. The reason why those bars are missed is due to the weekend-filter which essentially filters all data after what it considers the close of market (which in Dukascopy data is Friday 22:00h). We can see the issue that it could cause for you and will try address it with a few more options. I believe the original reason for the filter was there was some "extraneous" tick data in the Dukascopy feed occurring after market close that was causing an issue for some users. For the moment, you should be able to "include weekends" which should reveal those missing bars in the exported data.

Hope this helps.