• Purpose of This Blog and Information about the Author

Latticework Wealth Management, LLC

~ Information for Individual Investors

Latticework Wealth Management, LLC

Tag Archives: correlation

Breakthrough Drugs, Statistics, and Anecdotes: Three Things Every Individual Investor Needs to Know – Statistics and Time Series Data – Part 2 of 3

20 Wednesday Nov 2019

Posted by wmosconi in asset allocation, correlation, correlation coefficient, finance theory, financial advice, Financial Media, Financial News, financial services industry, historical returns, Individual Investing, individual investors, investing, investing advice, investing tips, risks of stocks, standard deviation, stock market, Stock Market Returns, time series, time series data

≈ 3 Comments

Tags

correlation, correlation coefficient, Financial Market History, financial markets, Financial Media, Financial News, invest, investing, investing tips, math, mathematics, noise, statistics, time series, time series data

The first article of this three-part series covered the broad strokes of this issues to be aware of in terms of all the “data” and “relationships” that get thrown around by the financial media (print and television).  Most of the discussion uses data points that are not statistically significant to draw any sort of conclusion.  In fact, time series data is notoriously hard to model and predict the future.  Additionally, the specific time series data of stock market returns is even more difficult.

You can refer to the link below to examine the content of the first article:

https://latticeworkwealth.com/2019/11/11/breakthrough-drugs-statistics-and-anecdotes-investing/

The task at hand for the second article is to put some “meat on the bones” of the discussion.  I realize that anything to do with math and statistics is not easy for everyone (or of interest either).  Therefore, I will be writing a supplemental article that covers the mathematics and statistics in more detail.  The goal here is to be able to identify some of the more common errors that you will encounter.

The first item to talk about is any sort of data that has a substantial trend component.  In layman’s terms, there is a data series where the line graph goes up or down in more of a straight-line manner.  You can think of the Gross Domestic Product (GDP) of the United States here.  Every year the GDP figures will generally go up unless there is a recession.  But, even after the recession passes, the trend for GDP will resume upward.  So, where does the problem come in?

I am going to give a contrived example to illustrate why it is dangerous to compare two series that are trending.  The example will consist of two different equations which are trends.  Both have the same trend component and an error term (we will call that eta).  The variables will be exactly the opposite.  More specifically, the two equations we will use are the following:

Trend_1 = Time + 100 + 0.9 * x + eta

Trend_2 = Time +100 – 0.9 * x + eta

Now the x values and eta values were simply generated by selected variables at random between 0 and 1.  The eta values were also selected at random between 0 and 1.  You can think of eta as representing the general “noise” that occurs on a daily basis when observing stock prices in the financial markets.  So, let’s graph the first 100 observations for these two equations:

Trend_Graph_Statistics_Revised

You will notice that the trend component dominates the line graphs.  However, we know by construction that the two equations which produce trend_1 and trend_2 are fundamentally different.  Now the correlation coefficient between those two equations is 0.9984.  A correlation coefficient of 1 means that the two lines move in lockstep.  Why is this important?  Why is it very dangerous?

Well, financial pundits will talk about these types of graphs all the time.  It looks like there is some relationship, but we know there is very little relationship between the two trends.  In fact, we can look at these equations by subtracting the current value from the previous value to see what changes.   Formally, this topic is called first differencing.  It will allow us to see more clearly what we already know.  Here is the graph:

First_Difference_Graph_Statistics_Revised

Now we have a totally different picture.  We can see that at many times the two trend equations are moving in exactly the opposite direction.  In fact, the correlation coefficient for the first-differenced equations is 0.2675.  There is only a slight positive relationship between the two trends.

In the example above, we can see that looking at the two trends is very deceiving.  Remember that I added the eta term to represent “noise” that is always present in financial market data.  So, anytime someone talks to you about the comparison of two trends, you should be very skeptical.  You always want to see first-differenced data or at least a comparison of changes in some manner.  Otherwise, you will mistakenly assume that there is a strong positive or negative relationship between two time series.

The second example that I am going to use is stock market returns for the S&P 500 Index from 1966 through 2018.  Why start at 1966?  Well, the S&P 500 Index started with its current number of component stocks back in 1957, and I would like to show annual stock returns and also ten-year annualized returns.  This particular topic can get messy quite quickly, so I am not going to cover it in a lot of depth with statistical and mathematical jargon.  For those of you who are interested, I had mentioned that it will be contained in a forthcoming supplemental article.

A great many individuals in the financial markets talk about stock market returns in the same breath as the normal distribution.  What is the normal distribution?  It is the old bell curve that you are familiar with.  The normal distribution is symmetrical and tails off at the end as more and more data points are gathered.  Well, stock market returns are anything but strongly normal.

Let’s first take a look at one-year stock market returns for the S&P 500 Index.

One Year Returns - Histogram - Non Normal

A useful test to see if a particular distribution is normal is the Jarque-Bera test.  Now it is not necessary to know exactly what is being calculated.  However, you should refer to the bottom of the box that reads “Probability”.  The value of 0.179 is called a p-value.  A p-value less than or equal to 0.10 means that we can reject the hypothesis that the one-year distribution of stock returns is normal.  At a value of 0.179, we would not reject the hypothesis of normality for this distribution.  However, the p-value in our case is not large enough to be totally sure and confident. But what about looking at annualized stock market returns over ten-year periods?

We can look at a similar graph to check to see if stock market returns over longer timeframes are indeed akin to the normal distribution (i.e. the bell curve).  Here is the graph:

Ten Year Returns - Histogram - Non Normal

Looking at the same “Probability” value, we have 0.489.  Therefore, we cannot reject the hypothesis that these stock market returns follow a normal distribution. Looking at ten-year annualized stock market data tells us that we can use the normal distribution as an assumption for calculating statistics.

Now why does this matter?  Well, you will here over and over again statistics that apply only to the normal distribution in relationship to actual, observed stock market returns.  We have just seen that stock market returns over the short-term stock returns weakly follow the normal distribution. On the other hand, long-term stock returns are definitely normal. Now I will not get into the technicalities, but time series data is indeed asymptotically normal.  What?  Say again?

This is just a fancy way of saying that, as the number of data points (sample size) approaches infinitely, the time series will look like the normal distribution.  Pretty much all financial market and economic data have very few data points.  In fact, you usually need several hundred data points prior to making any assumptions and using the statistics related to the normal distribution (think standard deviation or correlation coefficients).

Thus, most of the banter in the financial media is just subjective notions of what is going on in the stock market and the economy. More often than not, an assertion by someone in the financial print or television media is more of an educated guess than based on a solid mathematical foundation. That fact explains why financial pundits hedge their statements. Like I say half-jokingly, “I see the stock market going up in the next several months, but of course it might not resume its uptrend or could even take a leg downward”.

Well yes, I guarantee you that every day stocks will go up, down, or remain unchanged. This type of daily commentary in the financial press about the short-term performance of stocks (or other financial assets) is just not helpful and can be downright distracting you from investing for your long-term financial goals.

I apologize for getting too detailed in certain parts of this article.  What are the key takeaways?  First, you should be extremely leery of drawing any conclusions from the comparison of two or more data series that are trending upward or downward.  Second, you need to have several hundred observations prior to invoking any reference to the normal distribution.  So, what is left after that?  As you might imagine, there are not too many comparisons or studies that pass the muster to give you insights on investing or actionable information to make changes to your investment portfolio.

Don’t focus on the mathematics or statistics.  All you need to remember are the two takeaways above.  And, first and foremost, you should always be skeptical whenever you are presented with comparisons and statistics related to the financial market or the economy as a whole.

How to Become a Successful Long-Term Investor – Part 3 of 3 – The Folly of Market Timing

28 Saturday Sep 2019

Posted by wmosconi in Alan Greenspan, asset allocation, Average Returns, behavioral finance, bubbles, correlation, correlation coefficient, Dot Com Bubble, finance, finance theory, financial goals, financial markets, Financial Media, Financial News, financial planning, Greenspan, historical returns, Individual Investing, individual investors, Internet Bubble, investing, investing advice, investing information, investing tips, investment advice, investments, Irrational Exuberance, market timing, math, personal finance, portfolio, risk tolerance, risks of stocks, S&P 500, S&P 500 historical returns, S&P 500 Index, statistics, stock market, Stock Market Returns, Stock Market Valuation, stock prices, stocks, Valuation, volatility

≈ 1 Comment

Tags

asset allocation, behavioral finance, bubbles, correlation, correlation coefficient, finance, invest, investing, investing blogs, investing strategies, investing tips, investment advice, investments, long term investing, long-term investor, market timing, math, mathematics, portfolio, statistics, stocks, successful investor, trading, uncertainty, volatility

This article is the third and final post in my three-part series on learning how to be a successful long-term investor.  The general theme underlying all of the topics has been developing enough of an understanding of the stock market gyrations and sometimes wild ride to form reasonable expectations at the outset.  Those expectations lead directly into to developing a long-term investment strategy and plan that you are much more likely to stick with through “thick and thin” because you know what is coming.  Of course, you will not know the order in which the ups and downs may come, but you will have a ton of information helpful to be much less likely to lose your nerve or get overly excited.

The last topic will be about “market timing”.  We will delve deeply into the concept and see how very difficult it has been in the past, and, I believe, will continue to be for the foreseeable future.  Now the discussion to follow will be entirely self-contained; however, it might be helpful to take a look at the first two articles to have additional context.  The opening topic was an overview of the history of stock market returns using the S&P 500 Index (dividends reinvested).  Here is a link to that post:

https://latticeworkwealth.com/2019/09/23/successful-long-term-investing/

The second topic was a discussion about the concept of risk.  We explored how it is normally defined, ways that you can gauge your tolerance for risk given the information from the first post, and explored some methods/mindsets to reduce risk in your investment portfolio.  Here is a link to that post:

https://latticeworkwealth.com/2019/09/25/successful-long-term-investor-risk/

So now, we will turn to the topic for the last article.  As mentioned above, we are going to take a look at “market timing”.  In general, the idea of “market timing” is to develop ways to be able to buy stocks when they are very undervalued and also sell stocks right near the market peak to avoid a big downturn.  There are certain variations where an investor is not necessarily trying to time the most opportune time but trade along with the momentum of the stock market and anticipating the next movement prior to other stock market participants.

“Market timing” is notoriously difficult to do.  But you will see considerable time devoted every day to financial market television and periodicals advising individual investors what trades to make.  I would submit that following things and pundits on a daily basis adds to “noise” and “information overload”.  Additionally, for every guest that predicts a big leg up in the market, there will be another guest later in the day who tells you that we are in a bubble and stocks will drop dramatically soon.

Another lesser talked about item is the main guests that are invited to speak on television or are quoted in financial periodicals.  Typically, the guest introduction will be prefaced by this man/woman predicted the last major move in the stock market and we are so lucky to have him/her back again.  While these guests are great to hear from, there is a severe amount of “selection bias”.  What do I mean by “selection bias”?  You will rarely see a guest brought on to be lambasted for a prediction that never came to fruition or was just flat out wrong.  The vast majority of guests on television or market experts in financial articles will be the ones who made a very prescient call on the direction of the stock market.

The promise of “market timing” is still so enticing.  It normally relates to the fear of losing money or the greed of just not wanting to miss the next big bull market trend upward in the stock market.  However, the ability to call the market tops or bottoms has proven to be pretty much a 50/50 flip of the coin (now I am being generous at that).  One of the examples that I love to give is the coining of the term “irrational exuberance”.  The former chair of the Federal Reserve, Alan Greenspan, used that new term to state that the stock market was in what he thought was a bubble.  Little do people remember, but he first gave the speech in December 1996 to refer to what would become the Dot.Com bubble and bust.  Greenspan was proven right but the top of that bubble occurred in March 2000.  I use that example because irrational activity in the markets can persist for much, much longer than you might expect.

So, now I know that some people reading this post will be able to point to experts who made the great calls or even their own calls on the direction of the stock market.  Well, I will start off the discussion by showing that “market timing” is indeed somewhat possible.  But it takes much longer periods of time than you might think at first.  Here is how we will proceed in the analysis.  I discussed how the long-term historical average of the S&P 500 Index from 1957-2018 has been 9.8%.  It would seem logical then that, if stock market returns were below that average or above that average for a certain length of time, you could just do the opposite figuring that stock market returns would eventually trend back to that average (in the jargon reversion to the mean).

The problem is, as I briefly mentioned in the last paragraph, that the time period needs to be so long that it is almost untenable for individual investors to practically implement.  In fact, we have to use 15-year annualized returns to illustrate the theory.  So, if the stock market has been below/above trend, we will buy/sell because an inflection point has to come.  Let’s take a look at it graphically to drive the point home:

Fifteen Year Correlation

In the graph depicted above, we have exactly the returns we would like to see.  The blue dots are the past 15 years of stock market returns, and the orange dots are the next 15 years of stock market returns.  The dots are what we would term to have an inverse relationship.  In fact, for all of you somewhat familiar with statistics, the correlation coefficient is -0.857.  Therefore, there is a really strong relationship here that leads us to the promise of “market timing”.  Should we give up on it so early?

The problem with “market timing” is that, for any length of time less than 15 years of annualized stock returns, there really is no relationship (at least no actionable trading of stocks for your investment portfolio).  Let’s take a look at the same concept in the first graph with a look at one-year and three-year current and then future returns:

One Year Correlation

Three Year Correlation

Using the one-year and three-year current and then future stock market returns of the S&P 500 Index, our dots just kind of do not follow a discernable pattern.  Again, for the statistically inclined folks out there, the correlation coefficients are -0.10 and -0.041, respectively.  As always, we won’t get too waded down into the mathematical weeds but a correlation coefficient close to 0 means that there is essentially no correlation/relationship between the two.  To make an analogy, you can think of what is the correlation between birds in your backyard and the number of jars of pickles for sale at your local grocery store?  Well, there should be no relationship whatsoever.  Even if there were, it would not make any sense.  In our case here, there is at least some logic underlying our premise of the most recent return on the S&P 500 Index and the future returns over that same time period.  As we see though, there is really nothing actionable to embark upon for individual investors to properly engage in “market timing”.

Before we totally give up on “market timing”, we can take a look at the same charts but extending the time periods to five years and ten years.  Let’s take a look at those two graphs:

Five Year Correlation

Ten Year Correlation

The correlation coefficient for the five-year chart is 0.028, so we cannot really use that long of a time period either.  I will admit that the ten-year chart looks a little more promising.  We have a graph that looks somewhat more like the fifteen-year graph that I started off with.  In fact, the correlation coefficient is -0.276.  And a negative number is what we want to see in order to try “market timing”.  Unfortunately, the number is really not strong enough to not get caught.  By this I mean, we can see that “market timing” would have worked from 1975-1985 and also from 1990-2001 roughly.  However, 1965-1975 has a grouping of returns that don’t work and 2002-2008 has mixed results as well.  Note that there are less data points because there needs to be at least 10 years of future returns in order to compare the current record of 10-year annualized returns with what the next 10 years of stock returns will end up being.

Overall, we have seen that “market timing” in the short term (even as defined out to five years) does not really have much, if any, predictive power.  Therefore, if you make decisions related to “market timing” based upon how the stock market has performed in any time period five years or less, it is clearly a “fool’s errand” or incredibly difficult to do.  And by the latter, I mean that you can reliably do so over more than one major change in market direction.  The majority of market pundits that you will see or read about have made one correct call which is not nearly enough to judge his/her investing acumen related to “market timing”.

I will close out the discussion of “market timing” by using the Financial Crisis and ensuing Great Recession.  Many folks correctly called (or were proven right without the reason for the bubble matching their investment thesis) this major stock market inflection point.  They correctly saw the unsustainable bubble in housing, the rise of financial stocks, and the buildup of toxic securities like subprime loans.  However, many of those same individuals never changed their investment thesis and failed to tell individual investors to return to the stock market and buy.  Essentially there are still folks that will tell you we are in a bubble.  Now I am not bold and/or grandiose enough to weigh in on the current value of the stock market.  But you need to know that most of the people who call a wicked crash in stocks or a massive bull market do not change their investment thesis prior to the next big turn.

For example, let’s say that you learned about stock investing 10 years or so ago and decided to invest $1,000.00 in the S&P 500 Index toward the end of October 2007.  And yes, this was the absolute worst time to invest in stocks.  Sadly, by March 2009, you would have lost 50% of your investment and have only $500.00 at that point in time.  You might feel great if you listened to someone who called the top and told you that the fourth quarter of 2007 was the absolute worse time to buy stocks.  But I am willing to bet that this same person would not have told you when it was “safe” to invest again.  If you knew to expect bouts of extreme volatility in the stock market beforehand, you could have kept your money in the stock market.  At the end of December 2018, you would have had $1,712.36 using our 13.1% 10-year annualized return over that time.  If the original market predictor of catastrophe told you to just keep your $1,000.00 in the bank you would have $1,160.54 (assuming generously that you could earn 1.50% over the ten years in your bank saving account).  Adjusting the hypothetical investor who simply kept his/her money in stocks back to inflation, he/she would have $1,404.73 (assuming 2.0% inflation over the last 10 years which is higher than was actually experienced).  At the end of December 2018, you would have a bit more than 21% higher in inflation-adjusted dollars than the person who just never invested (or took his/her money out of stocks right at the end of October 2007 but never returned to stocks).

Now I will admit that my hypothetical scenario would have tried the “intestinal fortitude” of the most seasoned professional investors after seeing a 50% market drop over 1.5 years.  My only point with the example is that, even if you could not have held your nerve to remain invested in stocks over the Financial Crisis, the investment pundit(s) who tells you the exact top with a brilliant prediction also needs to tell you when to invest or sell again in the future (i.e. “market timing”).  Rarely will you see such a prognosticator that can totally change their investment thesis to get the next call right.  You are much better off abstaining from “market timing” and sticking to your long-term investment strategy.  Of course, that may indeed call for selling or buying a portion of stocks at certain given points to change your investment portfolio allocation to match your risk tolerance and financial goals.  But trying to utilize “market timing” to be in and out to experience hardly any losses and capture all the gains is just not realistic, so you might as well discard the entire investment strategy of “market timing”.

Subscribe

  • Entries (RSS)
  • Comments (RSS)

Archives

  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • April 2017
  • July 2016
  • May 2016
  • March 2016
  • December 2015
  • November 2015
  • July 2015
  • June 2015
  • May 2015
  • August 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013

Categories

  • academia
  • academics
  • active investing
  • active versus passive debate
  • after tax returns
  • Alan Greenspan
  • alpha
  • asset allocation
  • Average Returns
  • bank loans
  • behavioral finance
  • benchmarks
  • Bernanke
  • beta
  • Black Swan
  • blended benchmark
  • bond basics
  • bond market
  • Bond Mathematics
  • Bond Risks
  • bond yields
  • bonds
  • book deals
  • books
  • Brexit
  • Brexit Vote
  • bubbles
  • business
  • business books
  • CAPE
  • CAPE P/E Ratio
  • Charity
  • Charlie Munger
  • cnbc
  • college finance
  • confirmation bias
  • Consumer Finance
  • correlation
  • correlation coefficient
  • currency
  • Cyclically Adjusted Price Earnings Ratio
  • Dot Com Bubble
  • economics
  • Education
  • EM
  • emerging markets
  • Emotional Intelligence
  • enhanced indexing
  • EQ
  • EU
  • European Union
  • Fabozzi
  • Fama
  • Fed
  • Fed Taper
  • Fed Tapering
  • Federal Income Taxes
  • Federal Reserve
  • Fiduciary
  • finance
  • finance books
  • finance theory
  • financial advice
  • Financial Advisor
  • financial advisor fees
  • financial advisory fees
  • financial goals
  • financial markets
  • Financial Media
  • Financial News
  • financial planning
  • financial planning books
  • financial services industry
  • Fixed Income Mathematics
  • foreign currency
  • forex
  • Forward P/E Ratio
  • Frank Fabozzi
  • Free Book Promotion
  • fx
  • Geometric Returns
  • GIPS
  • GIPS2013
  • Greenspan
  • gross returns
  • historical returns
  • Income Taxes
  • Individual Investing
  • individual investors
  • interest rates
  • Internet Bubble
  • investing
  • investing advice
  • investing books
  • investing information
  • investing tips
  • investment advice
  • investment advisory fees
  • investment books
  • investments
  • Irrational Exuberance
  • LIBOR
  • market timing
  • Markowitz
  • math
  • MBS
  • Modern Portfolio Theory
  • MPT
  • NailedIt
  • NASDAQ
  • Nassim Taleb
  • Nobel Prize
  • Nobel Prize in Economics
  • P/E Ratio
  • passive investing
  • personal finance
  • portfolio
  • Post Brexit
  • PostBrexit
  • probit
  • probit model
  • reasonable fees
  • reasonable fees for financial advisor
  • reasonable fees for investment advice
  • reasonable financial advisor fees
  • rebalancing
  • rebalancing investment portfolio
  • rising interest rate environment
  • rising interest rates
  • risk
  • risk tolerance
  • risks of bonds
  • risks of stocks
  • Robert Shiller
  • S&P 500
  • S&P 500 historical returns
  • S&P 500 Index
  • Schiller
  • Search for Yield
  • Sharpe
  • Shiller P/E Ratio
  • sigma
  • speculation
  • standard deviation
  • State Income Taxes
  • statistics
  • stock market
  • Stock Market Returns
  • Stock Market Valuation
  • stock prices
  • stocks
  • Suitability
  • Taleb
  • time series
  • time series data
  • types of bonds
  • Uncategorized
    • investing, investments, stocks, bonds, asset allocation, portfolio
  • Valuation
  • volatility
  • Warren Buffett
  • Yellen
  • yield
  • yield curve
  • yield curve inversion

Meta

  • Register
  • Log in

Blog at WordPress.com.

Cancel

 
Loading Comments...
Comment
    ×