This post presents an appealing proof for the progressing domination of algorithmic trading over human trading. By analysing the US stock market between 1960 and 1990, we estimate a human engagement (human factor) in live trading decisions taken after 2000. We find a clear distinction between 2000-2002 “dot-era” trading behaviour and 2007-2008 crisis. By the use of peculiar data samples, we support their usefulness being motivated by recent discoveries in quantum physics helping us to better understand the physics of human thoughts and shared emotions when related to the same topic, here, the fear of losing.
1. Introduction
When I was 16 someone pointed my attention to the phenomenon of fear spreading across the room. The example was taken (nearly) from a daily life. Namely, imagine that you are sitting in the IMAX cinema’s room. It’s dark and all people are fully absorbed by a new 2.5-hour long action movie with James Bond in 3D. There is a lot of tension in the air. Suddenly, out of the blue, someone stands up and screams loudly: There is a snake here!! What happens next??! There is a panic! And… it’s spreading instantaneously across the room! How is it possible?!
Believe me or not but I devoted the past 20 years of my life to the investigation of this unbelievable event. Well, from the sidelines it looks normal, right? A snake in a dark cinema’s room is a rare occurrence indeed however it is not about the cinema! It is all about the imprinted information in our brains: the knowledge about snakes. They are fast, dangerous, and (in most cases) deadly venomous! That’s all what we know about them leaving an elementary school. That’s all we need to know in order to create a firm association in our minds that “a snake” means “it will kill you”! That’s it. End of the story.
When I was 18, some people were telling me some stories about human’s aura. A “mystic” energy-like field surrounding our bodies. Permanently. If you know how to interact with it, you can influence it in a good or bad way. You can help in healing or interrupt the flow of “chi” energy around the body. At that time, it was more (than ever) mind-blowing piece of knowledge handed me without any proof of its validity. I guess you know what I mean by that. When we are young, we assume so many “facts” by faith. A few dare to explore them. Deeper.
Science is a field of study where it takes an event as it is and seeks for the ways, methods, approaches in order to prove or discard its truthfulness. Before the humankind invented an Infra-Red (IR) camera, people had no idea about an IR imaging of our bodies. With seeing “outside the box” we proved (once and for all) that our body’s emission extended beyond the visual light within which one our eyes could “see” the world around. Therefore, what is invisible and undetected, it does not mean that it does not exist. Sometimes, discovering it is a matter of time. Sometimes, it’s a matter of turning the impossible into “faith” that it is possible.
Humans. We are made to have and share emotions. What are they, for example, an emotion of fear? If you ask a neuroscientist he/she will tell you: it’s a past experience (therefore a new block of knowledge) coded in and by your brain. Once anything similar is “experienced again” by you, your brain will seek for a relevant information and react “as previously”. With every brain’s activity (not solely related to the human’s emotions) the corresponding reaction is engaged via chemical, neurological, and muscular feedback. If the response to “a new event” in your life has to be made, you learn by “hit and miss” method. If you know “what to do”, you are more “experienced” (or your mind’s processes trigger an intellectual thinking leading you (more likely) to finding the “best” solution to the existing problem or situation). The gravity of our “thinking” becomes exponentially complex the more you go into it. Shortly you discover an astonishing fact about yourself: the more you explore yourself, the less you understand yourself!
When I was 20, I studied quantum mechanics at uni. I spent nearly 6 months looking at a bizarre bra-ket notation of quantum states without a clear understanding what the lecturer was talking about. But a dead Schrodinger’s cat was not so dead as it appeared to be. I came back to the quantum mechanics in my mid-30s, backed by PhD in astrophysics, and with my relentless determination to seek and find the answers to the most fundamental questions related to the nature of humans’ mind way of thinking and interpersonal communication.
When I was young, someone taught me all about making my dreams a reality. The formula was staggering: close your eyes and imagine yourself as a person who already has achieved it. Damn straightforward… And it all occurred to work that way. I followed the rules, I worked hard over my life and my dreams (e.g. of living one day in Sydney, Australia) and… now I live exactly where I dreamt to live.. in Sydney, Australia! Not in Mozambic nor Buenos Aires.
That was a big shot. Plenty of “impossible” events took place along the way…
I used to read a lot. I was deeply surprised what I found in the words of Jesus Christ. Again, I’m an amateur when it come to the Bible. I’m not a Pro. But what I can quote here I think will be sufficient: “whatever you ask for in prayer, believe that you have received it, and it will be yours” (Mark 11, 24). The Present Perfect tense: that you have received it. That was a Jesus’ confirmation on what today people teach: close your eyes and imagine you are already living your dream and it will soon become your reality. Period. We find the same message in the book of Wallance D. Wattles The Science of Getting Rich (1910) and in the books of Napoleon Hill (1930s) just to name the best ones.
How is it possible that emotions and feelings mixed with our inner vision of “reality” seek and find their way in real life? Do we really attract them or it is a playground of “God” who only knows the equation of state and “does” the miracles? Or maybe, we, humans, possess the powers of mind, the potential within that we barely tap on.
The past 50+ years of developments in science, philosophy, religions, and brain research started to turn our understanding of “unknown” processes towards one common denominator in this game: the quantum mechanics. The bizarre one.
2. Quantum Mechanics of Human Thoughts: Quantum Entanglement
The early years of the 20th century were the cornerstone of quantum physics and battlefield between it and the classical physics. Bohr against Einstein. Einstein did not believe that God played a dice. The randomness built-in at micro scales frightened him. The birth and progress in quantum mechanics (QM) introduced a new package of uncertainties: a vision of the Universe ruled by the laws going against our classical logic, remaining counter-intuitive.
The QM has soon been confirmed to be valid in the observational world of wonders. Just recall a two-slit experiment with an electron passing through. A mechanical wave on the water interferes. The electron (or any other particle) doing the same triggers the interference patterns however with a completely different distribution of “minimas” and “maximas” when contrasted with a classic physics’ expectations. By observing a wave on the water, we clearly see the effect of “going through” the slits. In quantum world, we have no idea through which slit the particle chose to penetrate.
Quantum Mechanics delivers a super-complex picture of reality at the fundamental level. The movement of a particle in space-time (4D) could be described as a “wave”. A special kind of wave, far different from our “classical” comprehension. Technically speaking we associate so-called wave function, $\Psi$, with moving or interacting particles. The wave function is a solution to the Schrodinger equation:
$$
\hat{H}\Psi(x,t) = i\hbar \frac{\partial \Psi}{\partial t}
$$ which describes the time evolution of a particle. How bizarre this is one can already see by the presence of an imaginary number of $i$ “built-in” into equation.
The most widely accepted interpretation of $\Psi$ reveals its dimensionless nature to be a wave of probabilities spreading “across the room”. It is impossible to see or measure $\Psi$ in a quantum world. Once the observation is done, for example, of a moving electron or photon by a detector, the whole wave function collapses instantaneously. We can derive the amplitudes of probabilities by:
$$
|\Psi(x,t)|^2
$$ but never be certain where the particle is located before the observation.
$\Psi$ is an abstract mathematical object living in the Hilbert space. It represents a particular pure quantum state of a specific isolated system of one or more particles. By choosing a specific system of coordinates, e.g. $\xi = (r, \vartheta, \psi)$ or $\xi = (x,y,z, \sigma_z)$, one can represent the wave function of $\Psi(\xi)$ as:
$$
\Psi(\xi) = \left( \Phi_{\xi}, \Psi \right) = \left( \Phi_{(x,y,z,\sigma_z}), \Psi \right)
$$ i.e. the inner product of a state vector $\Psi$ and a state $\Phi(\xi)$ in which the particle is located in 3D by $x$, $y$, $z$, having a definite value spin $z$-projection of $\sigma_z$. For example, the wave function for a hydrogen atom in different quantum states in spherical coordinates is given by:
$$
\Psi_{n,m,l}(r, \vartheta, \psi)
$$ i.e. depends on quantum numbers (for a full formula see here). The visualization of probability density for Hydrogen (see here) provides us with a better “feeling” where we can find its electron in a given state. But again, we never know where the electron is until the moment of observation.
A wave function for $N$ particles in 4D space-time can be denoted as:
$$
\Psi(\textbf{r}_1,\textbf{r}_2,…,\textbf{r}_N, t)
$$ where $\textbf{r}_i\equiv \textbf{r}_{(x,y,z), i}$ are the position vectors for all $i=1,2,…,N$ particles at time $t$. Interestingly, for $N$ particles in QM there are no $N$ wave functions (for each particle separately) but rather one wave function describing them all. Moreover, that leads to the phenomenon of Quantum Entanglement — a physical phenomenon that occurs when pairs or groups of particles are generated or interact in ways such that the quantum state of each particle cannot be described independently — instead, a quantum state must be described for the system as a whole. That’s very important as we will discuss it, shortly.
What is a human thought? Technically speaking it is an electrical impulse generated in the brain. Where does it originate exactly? It’s difficult to say. We have no way or method to capture the birth place of a thought, for example, when you decide to think about a white ping-pong ball. With a help of ElectroEncephaloGraphy (EEG) we can measure the electrical activity of the brain. EEG measures voltage fluctuations resulting from ionic current within the neurons of the brain. On the other hand, the functional Magnetic Resonance Imaging or functional MRI (fMRI) is a functional neuroimaging procedure using MRI technology that measures brain activity by detecting changes associated with blood flow. Both approaches, though very useful in medicine and neurobiology, leave a vast room for “capturing” our thoughts.
An electrical impulse sent by the brain commences a whole avalanche of neuro-chemical-muscular processes in our body. It is super-complex. No doubt. In 2004, the team of neurobiologists lead by Massimo Scanziani from University of California San Diego have uncovered evidence that sheds light on the long-standing mystery of how the brain makes sense of the information contained in electrical impulses sent to it by millions of neurons from the body. We are bombarded by thousands of information every second. Somehow, our brain does an amazing job for us. How information is sorted by the brain has been an open question. The group discovered that different neurons in the brain are dedicated to respond to specific portions of the information (see a full report here). But what is a human thought? Just an electrical impulse or a series of impulses in a specific “state”?
That’s where quantum mechanics may help us in building a clearer picture, therefore turning metaphysics into real physics. If, technically considering, a thought emerging from our “intellect” or “consciousness” can be associated with a series of elementary particles (e.g. electrons, etc.) “running” through our bodies, is it possible to allow them “spreading across the room” or “around the world” or even “around the Universe”?! I.e. to leave “us” (the body which we consider as something that usually defines us in the visible band of the EM wavelengths) and interact with other(s) at great distances?
It seems not to be a science-fiction any more. A particle in a quantum world may “tunnel” through the wall what in a world we know is impossible. You cannot walk through the Great Wall of China through the bricks of stone (unless your name is David Copperfield and you did it; see the movie if you missed that; a great magical trick with light and stairs).
The hypothesis of quantum tunnelling taking place in our bodies has been formulated by Turin in 1996. “Turin posited that the approximately 350 types of human smell receptors perform an act of quantum tunneling when a new odorant enters the nostril and reaches the olfactory nerve. After the odorant attaches to one of the nerve’s receptors, electrons from that receptor tunnel through the odorant, jiggling it back and forth. In this view, the odorant’s unique pattern of vibration is what makes a rose smell rosy and a wet dog smell wet-doggy.” — we read in Discover Magazine (2009). Quantum tunneling has also been observed in enzymes, the proteins that facilitate molecular reactions within cells; the papers published in 2006 and 2007 in Science and Biophysical Journal, respectively.
If quantum tunnelling is taking a place as a part of “us” being alive, it may be the fundamental indication towards understanding how our thoughts spread out in and out. Again, this is at the abstract level of our understanding of “what’s going on” and we are forced only to speculate.
Nevertheless, a human thought may be viewed in the quantum world as a time-dependent evolution of (charged) particles in a form of the wave function in a given quantum state. It is tempting to postulate, therefore I dare to do it here, that a concentrated thinking (i.e. when we focus on one task only or we feel the emotion of fear/happiness/etc.) is the brain activity “releasing” the groups of particles that their quantum state must be described as a whole — quantum entanglement. If so, this is where the magic begins!
Say, if two people “feel” the same way, through the quantum entanglement of their quantum state(s), the wave function describing similar thought process,
$$
\Psi(\textbf{r}_1,\textbf{r}_2,…,\textbf{r}_N, t)
$$ must be in the quantum world a good representation of “shared emotions” or “shared thinking” about the same thing. According to the above formula, the more people “think” or “share” the same emotions, at the quantum level they become entangled. If true, that would define emotional quantum entanglement in a more physical way.
It’s nearly impossible to prove that QM works that way. However there are empirical methods that one can use to convince oneself that feeling “something” works regardless the distance. For example, a mother seems to be linked tightly with her child. If the child is locked in its room out the sight, reach, and event acoustically-separated and something is happening to it, the mother magically “sense” that state and reacts. Another example. If you find yourself among a group of people who stand or sit in silence, you quickly “sense” that there is something, e.g. wrong “in the air”. Such experiments were taken in the past. Your ability to “detect” the feeling is phenomenal. The groups sitting in silence were asked simply to think intensively about something horrible that took place in their lives. One more? You start thinking about someone who you wish were where you are. But you are in Chile and that person is absolutely out of your place. You receive a phone call from him in a matter of an hour (out of the blue!) and he claims that he had “that feeling” about you, and found the way to find and call you.
The number of similar examples is endless. Talking to flowers in order to help them bloom is my favourite one. Emitting different emotions and causing the change of structure in a frozen water have been investigated already by Russian scientists in 60s (see a full movie) and Dr Masaru Emoto Hado in 1994.
But we all know it from our daily life. There are people who you meet for the very first time and you “feel” great and positive from that very first contact. The “emit” positive vibrations. On the other side, when your spouse’s mother is a negative person, always complains and in her spirit wishes you not the best, you “sense” it. You start avoid her. Something triggers your fear or anxiety.
If the fear is an emotion spreading instantaneously (being or not governed by the unknown/already-known rules of quantum mechanics), we all become affected if we “resonate” at the same “wave”. The recent experiments of Xing-Can Yao et al. (2012) proved that we can entangle eight photons. The complicated instrumental setup they used to do so acted as a smart “brain” making this entanglement possible. What if, our brain is an extraordinary instrument that at the quantum level picks up, emits, and entangles a specific group of thoughts emitted by others in a given “emotional” state?! What if it works that way indeed…
3. Detecting Fear among Traders
Pretend it is 28th of May, 1962. You are a broker/trader pushing the deals over the phone. But that day is peculiar. The S&P 500 index opened at 59.15 which occurred to be also its maximum price (high). At the end of the trading session it closed the day at 55.50 and market dropped 6.17% in a single day. The Low was denoted at 55.42. The ratio between Open-and-Close to High-and-Low was over 97.5%. There was nearly no pressure to “buy”, i.e. strong supply vs low demand. People were selling stocks like crazy. The same did not happen again until 25th of October 1982 where the loss was nearly 4% at the ratio of 100% at over 8 times higher traded volume. What is going here?
If you watched the Wall Street movie or The Wolf of Wall Street, both included some scenes from an abnormal trading session where traders were affected by tension and nervousness due to falling stock prices. Well before 1990, the American stock markets’ order were conducted by people working in the brokerage firms. They were responsible for decisions made upon earlier agreements with their clients. As we have discussed earlier, the day when the market turns south causes people to think and act in a similar fashion. The feelings are spread across the floor and no one wants to climb the mountain while avalanche is coming down. A tidal wave that comes lifts all boats or put them down. We call that phenomenon “a human nature” but quantum physics may be involved in understanding why that is so.
Let’s consider S&P500 index. It is a good representation of US stock market behaviour, the sentiments or alertness to unusual trading factors. Assuming that between 1960 and 1990 the human engagement is trading decisions was dominant (over today’s computerized automatic order exectution), one may select from the historical data a peculiar data sample. Namely, again, we look for days when ratio between Open-and-Close to High-and-Low of S&P500 was over 95% and index closed with a loss. What we mean by that criterion is that we allow for 5% buying/selling pressure deviation (Open-High and Low-Close). Additionally, we use information on volume traded on those days. Why? Our second assumption relates to the fact that people (not computers) tend to display behaviour patterns fairly consistent over time. Therefore, the volume “pushed” on the days we seek for should remain pretty of the same magnitude (what we confirm in Figure 1 of our analysis below).
Using Python language and basic S&P500 data provider of Yahoo! Finance (the precision in prices is not a major concern here), first, we download the index prices and extract data sample based on abovementioned criteria:
# Detecting Human Fear in Electronic Trading # (c) 2016 Pawel Lachowicz, QuantAtRisk.com import numpy as np from scipy.stats import norm import matplotlib.pyplot as plt import pandas_datareader.data as web from scipy.stats.stats import pearsonr from scipy import stats data = web.DataReader("^gspc", data_source='yahoo', start='1960-01-01', end='1989-12-31')['Open','High','Low','Close'] data['OC'] = abs(data.Open - data.Close) data['HL'] = abs(data.High - data.Low) data['Ratio'] = data.OC/data.HL data['Loss'] = (data.Close/data.Open - 1)*100 # daily loss in percent events = data[(data.Ratio > 0.95) & (data.Loss < 0)] print(events.shape[0])
The analysis returns 71 days between 1960 and 1990. Not a lot but sufficient in order to draw some further steps.
4. Volume of Fears
The observation we dare to make here is directly linked to the examination of the relationship between traded volume on those days and a percentage loss. Again, we expect the volume to be somehow dependent of the magnitude of a daily loss but how strongly depended? Let’s derive the following:
pr, pvalue = pearsonr(events.Loss.values, events.Volume.values) slope, intercept, r_value, p_value, std_e = stats.linregress(events.Loss.values, events.Volume.values) print(pr, pvalue) # Pearson correlation coefficient print(r_value**2, p_value) # R-square value from a linear regression
where we find:
-0.789294146279 2.91365177983e-16 0.622985249351 2.91365177983e-16
pointing at pretty solid correlation though not super strong as intuitively expected. Based on the linear regression we plot this relationship:
plt.figure(num=1, figsize=(13, 6)) plt.plot(events.Loss, events.Volume, 'ro') x = np.linspace(-100, 0, 100) y = slope*x + intercept plt.plot(x, y, 'k:') plt.xlim([np.min(events.Loss), np.max(events.Loss)]) plt.ylim([np.min(events.Volume), np.max(events.Volume)]) plt.xlabel('Daily Loss [%]') plt.ylabel('Volume') plt.show() plt.savefig('fig01.png', format='png')
revealing:
What the linear fit does, it describes the simplest model between daily volume traded and, in good approximation, the fear in the market expressed by the loss on those days.
Since we made a remark on the amount of volume traded by people in those years, in the next Section, we will use it to the data recorded after 1999.
5. Disentangling Humans from Computers
We repeat the same procedure as outlined above but now for a data set covering the years 2000 to 2016:
del data, events data = web.DataReader("^gspc", data_source='yahoo', start='2000-01-01', end='2016-03-01')['Open','High','Low','Close'] data['OC'] = abs(data.Open - data.Close) data['HL'] = abs(data.High - data.Low) data['Ratio'] = data.OC/data.HL data['Loss'] = (data.Close/data.Open - 1)*100 events = data[(data.Ratio > 0.95) & (data.Loss < 0)] print(events.shape[0])
what reveals 147 days of the same character. Since it is logically justified that it only make sense to compare “apples” with “apple”, we build an estimation of human factor (HF; human engagement) is trading decisions after 1999 as:
$$
\mbox{HF} = \frac{ax+b}{y}
$$ or
events['HF'] = (slope*events.Loss + intercept)/events.Volume
where the coefficients $a$ (slope) and $b$ (intercept) have been found in the earlier linear regression analysis. $x$ and $y$ denote Daily Loss and Volume (taken between 2000 and 2016), respectively. In other words, $y$ is expected to be far higher “now” than “then” due to a global involvement of algorithmic automated (daily, intraday, and high-frequency) trading.
Therefore $\mbox{HF}$ ought to provide us with an idea on the volume traded solely by people and the rest by human-independent trading “robots”. By plotting the results:
ev1 = events[(events.index.year >= 2000) & (events.index.year <= 2002)] ev2 = events[(events.index.year >= 2007) & (events.index.year <= 2008)] plt.figure(num=2, figsize=(13, 6)) plt.scatter(100*events.HF, events.Loss, c='black', edgecolors='black', s=50) plt.scatter(100*ev1.HF, ev1.Loss, c='red', edgecolors='red', s=50) plt.scatter(100*ev2.HF, ev2.Loss, c='yellow', edgecolors='yellow', s=50) plt.legend(['2000-2016', '2000-2002', '2007-2008'], loc=4) plt.xlim([0, 25]) plt.ylim([-10, 1]) plt.xlabel('Human Factor [%]') plt.ylabel('Daily Loss [%]') plt.show() plt.savefig('fig02.png', format='png')
The result is spectacular. Not only we see a vast spread in human factors but also we learn that the lower limit on $\mbox{HF}$ extends up to 20% only. It may mean that from the year 2000 onwards, in over 80% the trading in the US market (as reflected by S&P500 index) have been dominated by algorithms.
In addition to that picture, we overplot with different colours $\mbox{HF}$s corresponding to 2000-2002 “dot-era” bubble (red markers) and 2007-2008 crisis (yellow markers). What is rather not so surprising is a clear boundary between both groups. During the last financial crisis, the selling of stocks have been executed by computer-based decisions in over 95% according to our findings.
6. End Note
Will computers eliminate human trading in not-too-distant future? Fear not. Just take a quantum leap of faith that it won’t happen soon…
4 comments
Hi Pawel,
There are such things as “inflation” and “economic development” – that’s why trading volume (is $ and probably in shares) should increase irrespectively of whether there are humans or robots who trade and of how emotionally they trade. Without accounting for inflation and overall increase of trading volume such sort of analysis is garbage.
Try to use not raw volume but its deviation (in %) from previous-quarter-mean, for example – results may change dramatically.
By introduction of complexity we spoil simplicity. “may change” is always an option, I do understand it. The quest is to do the global brainstorming in order to find (come up with) the best scenario that would describe the idea designed behind. And it’s complex too, I know…
Excellent analysis!
Thanks a lot!! :)