Simulation-based evaluation of automated trading strategies: a manifesto for modern methods

Abstract

In many investment banks and major fund-management companies, automated "robot" trading systems now do work that 20 years ago would have required large numbers of human traders to perform: the rise of robot traders is a major success-story for artificial intelligence (AI) research. Although the technical details of currently profitable automated trading systems are closely guarded commercial secrets, the rise of robot trading can be traced back to a sequence of key AI research papers. Each of these key papers relied on minimal abstract simulation models of real financial markets: the simulators provide test-beds for trials in which the performance of different trading strategies could be evaluated and compared. Recent studies have revisited these seminal results, using more realistic simulations of contemporary financial markets, and have cast major doubts on core conclusions drawn in the original publications. Therefore, it seems reasonable to argue that present-day simulation methods are exposing significant problems in past research on automated trading. This position paper presents no new empirical results but instead presents a review of key past papers and an argument, a manifesto, for establishing a shared market-simulator test-bed that adequately reflects current real-world financial markets, for use in future evaluation and comparison of trading strategies.

Financial markets | Automated trading | Simulation methodology 

References

1. Andrews, M., Prager, R., 1994. Genetic Programming for the Acquisition of Double Auction Market Strategies. in Kinnear, K. E. (d), Advances in Genetic Programming. MIT Press, pp.355-368.
2. Beyer, H.-G., Schwefel, H-P., 2002. Evolution Strategies: A Comprehensive Introduction. Natural
   Computing 1(1):3-52.
3. BSE, 2012. Bristol Stock Exchange. GitHub repo at
   https://github.com/davecliff/BristolStockExchange
4. Church, G., Cliff, D., 2019. A Simulator for Studying Automated Block Trading on a Coupled Dark/Lit Financial Exchange with Reputation Tracking. Proceedings EMSS2019.
5. Cliff, D., 1997. Minimal-Intelligence Agents for Bargaining Behaviours in Market-Based
   Environments. HP Labs Tech. Report HPL-97-91.
6. Cliff, D., Preist, C., 2001. Days Without End: On the Stability of Experimental Single-Period Continuous Double Auction Markets. Hewlett-Packard Labs Tech. Report HPL-2001-325.
7. Cliff, D. 2009. ZIP60: Further Explorations in the Evolutionary Design of Trader Agents and Online Auction-Market Mechanisms, IEEE Transactions on Evolutionary Computation. 13(1):3-18.
8. Cliff, D., 2018. An Open-Source Limit-Order-Book Exchange for Teaching and Research. Proceedings IEEE Symposium on Computational Intelligence in Financial Engineering (CIFEr); pp.1853-1860.
9. Cliff, D., 2019. Exhaustive Testing of Trader-Agents in Realistically Dynamic CDA Markets: AA Does Not Dominate. Proceedings ICAART2019.
10. Das, R., Hanson, J., Kephart, J., Tesauro, G., 2001. Agent-Human Interactions in the Continuous
    Double Auction. Proc. IJCAI-2001, pp.1169-1176.
11. De Luca, M., Cliff, D., 2011a. Agent-Human Interactions in the Continuous Double Auction,
    Redux. Proceedings ICAART2011.
12. De Luca, M., Cliff, D., 2011b. Human-Agent Auction Interactions: Adaptive-Aggressive Agents
    Dominate. Proceedings IJCAI-2011, pp.178-185.
13. De Luca, M., 2015. Adaptive Algorithmic Trading Systems. PhD Thesis, University of Bristol, UK.
14. Gjerstad, S., Dickhaut, J., 1997. Price Formation in Double Auctions. Games & Economic Behavior. 22(1):1-29.
15. Gjerstad, S., 2003. The Impact of Pace in Double Auction Bargaining. Working Paper, Department of Economics, University of Arizona.
16. Gjerstad, S., Shachat, J., 2007. Individual Rationality and Market Efficiency. Working Paper, Department of Economics, Purdue University.
17. Gode, D., Sunder, S., 1993. Allocative efficiency of markets with zero-intelligence traders. Journal of Political Economy, 101(1):119-137.
18. Hukerikar, N, 2019. Co-Adaptive Dynamics of Competitive Arms-Races Among Automated
    Trading Agents. Master's Thesis, Uni. of Bristol.
19. Le Calvez, A., Cliff, D., 2018. Deep Learning can Replicate Adaptive Traders in a LOB Financial
    Market. Proceedings of the IEEE Symposium on Computational Intelligence in Financial
    Engineering (CIFEr). SS1070, pp.1876—1883.
20. Miles B., Cliff, D., 2019. A Cloud-Native Globally Distributed Financial Exchange for Studying Real- World Trading-Latency Issues at Planetary Scale. Proceedings EMSS2019.
21. Rust, J., Miller, J., Palmer, R., 1992. Behavior of Trading Automata in a Computerized Double Auction Market. In Friedman, D., Rust, J. (eds) The Double Auction Market: Institutions, Theories, & Evidence. Addison-Wesley, pp.155-198.
22. Smith, V., 1962. An Experimental Study of Competitive Market Behavior. JPE 70(2):111-137.
23. Snashall, D., 2019. An Exhaustive Comparison of Algorithmic Trading Strategies: AA Does Not
    Dominate. Master's Thesis, Department of Computer Science, University of Bristol.
24. Snashall, D., Cliff, D., 2019 (in press). Adaptive-Aggressive Traders Don't Dominate. To appear in
    van den Herik, J., Rocha, A., Steels, L., (editors) Agents and Artificial Intelligence: Selected Papers from ICAART 2019. LNCS, Springer Verlag.
25. Stephens, C., Waelbroeck, H., 2009. Algorithm Switching: Co-Adaptation in the Market Ecology.
    The Journal of Trading, 4(3):59-73.
26. Stotter, S., Cartlidge, J., Cliff, D., 2013. Exploring Assignment-Adaptive Trading Agents in Financial Market Experiments, Proc. ICAART, 1:77-88.
27. Stotter, S., Cartlidge, J., Cliff, D., 2014. Behavioural investigations of financial trading agents using Exchange Portal (ExPo). In N. Nguyen et al. (eds), Trans. Computational Collective Intelligence XVII.
28. Tesauro, G., Das, R., 2001. High-performance Bidding Agents for the Continuous Double Auction. Proc. 3rd ACM Conf. on E-Commerce, pp.206-209.
29. Tesauro, G., Bredin, J., 2002. Sequential Strategic Bidding in Auctions using Dynamic Programming. In Proceedings AAMAS2002.
30. Toft, I., 2007. Adaptive Bidding Agents for Single Seller and Continuous Double Auctions. PhD Thesis, Computer Science, University of East Anglia, UK.
31. Vach, D., 2015. Comparison of Double Auction Bidding Strategies for Automated Trading Agents. MSc Thesis, Charles University in Prague.
32. Vytelingum, P., 2006. The Structure and Behaviour of the Continuous Double Auction. PhD Thesis, University of Southampton, UK.
33. Vytelingum, P., Cliff, D., Jennings, N., 2008. Strategic Bidding in Continuous Double Auctions. Artificial Intelligence, 172(14):1700-1729.
34. Witchett, D., 2004. Coadaptive Dynamics of Minimal Intelligence Trading Agents. Master's Thesis, University of Birmingham, UK.