Superforecasting For Investors: Part 2

Alpha Theory hosted a book club on December 6^th with portfolio managers, analysts, and allocators coming together to discuss “Superforecasting” by Phil Tetlock. We were lucky enough to have a Superforecaster, Warren Hatch, moderate and perform forecasting exercises with the group. We spent 2 hours together and only scratched the surface on applying Superforecasting to investing.
‍

Here are a few key takeaways:

1. COMMON ATTRIBUTES OF SUPERFORECASTERS:

INTELLIGENCE: Above average but genius isn’t required

QUANTITATIVE: Not only understand math but apply it to everyday life

FOXES, NOT HEDGEHOGS: Speak in terms of possibilities, not absolutes

INTELLECTUALLY HUMBLE: Understand the limits of their knowledge

SYSTEM 2 DRIVEN: Use the logic-driven instead of instinct-driven portion of their brain

DO NOT BELIEVE IN FATALISM: Life is not preordained

CONSTANTLY REFINE: Make frequent small updates to their forecast based on new information (but not afraid to make big changes when warranted)

COUNTERFACTUALS: Believe that history is one of many possible paths that could have occurred

OUTSIDE VIEW: Incorporate the internal and external views

GROWTH MINDSET: CONSTANTLY SEARCH FOR WAYS TO IMPROVE THEIR FORECASTING PROCESS

‍2. IDENTIFYING TALENT: There are identifiable attributes that can be used in hiring and have a profound impact on forecasting skill

Active Open Mindedness*

Fluid Intelligence*

* At a prior book club, we measured participants and the results showed they had the attributes of Superforecasters with high Active Open-Mindedness (3.99 out of 5) and high Fluid Intelligence (8 out of 10 – this is the highest score that the Good Judgment folks have seen).

Active Open Mindedness (i) and Fluid Intelligence (a) are two measurable traits that managers can use to select talent. In the chart below, the improvement impact of the definable attributes equates to about 40% of their forecasting skill over standard forecasts.

3. DEVIL’S ADVOCATE: Firms should appoint a Devil’s Advocate for each investment to expand critical thinking (someone to ask the question, “I see your downside is $40. How is that if the 52-Week Low is $22 and the trough multiple would put it at $25?”)

4. OUTSIDE VIEW: Firms should require an Outside View for every investment idea (“While everyone I’ve spoken to says this deal will close, only 20% of deals with one party under SEC investigation close.”

5. REFINEMENT: New information should always be incorporated in forecast (think Bayesian).

6. TEAMS MAKE BETTER FORECASTS: Team dialog generally improves forecasting accuracy.

7. FORECAST CULTURE: Firms should embrace “forecast” as part of their vernacular and conversations should revolve around how information impacts the forecast.

8. MEASURE TO BE BETTER: We all forecast, but we rarely measure. That fact needs to change if we really want to improve.

9. CLUSTERING: Break complex topics into individual components that are better able to be forecast and use the combination of the smaller forecasts to forecast the more complex. (ie. Will AAPL break $200 is a complex forecast that can be broken down into Will iPhone X ship more than 400m units? / Will Samsung’s technology outpace Apple’s? / etc.)

10. INDEXING: Individual clustering questions can be weighted to come up with a forecast for the complex topic instead of using simple equal weighting.

11. DIVERSITY OF FORECASTS MATTER: Forecasts made from similar perspectives are less accurate than those made from multiple perspectives (see Boosting below).

12. BOOSTING: If you have three forecasters with different perspectives that all arrive at a 70% probability of an event occurring then the actual probability is greater than 70%.

13. GISTING: We didn’t get to spend much time here, but the idea is that complex subjects, reports, presentations, etc. can be distilled down into gists that the team votes on and refines into supergist. Full understanding is never just quantitative or qualitative. Superforecasting is quantitative. Supergisting attempts to provide the qualitative piece.

14. HYBRID FORECASTING COMPETITION: IARPA, the defense agency that sponsored the forecasting tournament that launch the Superforecasters (Good Judgment) is sponsoring a new Man+Machine Forecasting Tournament. For those interested in Forecasting and Machine Learning, this is your spot: https://www.iarpa.gov/index.php/research-programs/hfc