Shinning light into dark places or muddying the waters of true compliance? A proper analysis of TJN’s methodology

The Tax Justice Network (TJN) published a “financial secrecy index” in 2013, to “shine light into dark places”to expose jurisdictions that support clandestine financial operations and tax avoidance. In supporting its efforts, TJN attacks the OECD’s report on tax co-operation, which it claims to be too “lenient”in measuring secrecy and tax-evasion.


The index has received media attention, including a recent article in The Economist. However, TJN’s analysis was not subjected to any type of rigorous outside review prior to publication, so obviously one may wonder about the validity of their approach.  Let’s break it down.


Suppose we give TJN the benefit of the doubt and assume they measured everything they want to measure with perfection.  All reporting parties are honest and nothing got lost in the mail. Unlikely to be sure, but we have bigger fish to fry here.


To start, how should such a study be conducted?  Several guidelines need to be met.  1).  In measuring “secrecy”, are an appropriate set of variables carefully selected, and are the variables scaled appropriately?  2).  Because the study wants to capture the effects of financial size, does it control for anything that may affect a jurisdiction’s financial export sector beside an attempt to irritate Uncle Sam and other tax collectors?  3).  Is the study robust to modest changes in potentially tenuous assumptions?


Let’s go to the TJN scorecard.


1).  Selecting and weighting appropriate variables.


TJN selects 15 “key”secrecy indicators.  Although there is little economic or statistical rationale given on why these indicators were selected, we’ll ignore this issue.  Perhaps more importantly, one would want to know how a given indictor really affects illicit behavior.  For example, is automatic information exchange between countries more important than the existence of bilateral tax treaties in mitigating tax avoidance/secrecy?  If so, by how much?


So, what rigorous analysis does TJN use in selecting the appropriate weights?  Difference-in-difference in a panel data setting?  Instrumental variables to control for potential endogeneity between financial size and secrecy?  Simple regression?  Magic eight ball?


The answer…NOTHING.  Absolutely nothing is done to control for the effects of the various indicators, as they are each weighted equally.  The study doesn’t even attempt to conduct any sensitivity analysis, by excluding/including certain indicators to determine if the results are robust.  This is especially strange when one considers that for one of the indicators related to banking secrecy, every jurisdiction receives the lowest identical score, effectively ensuring that all jurisdictions show a degree of secrecy, while biasing the study towards the size of the jurisdictions. For example, the same weight is given to any collection of beneficial ownership information (arguably the biggest secrecy driver) and to whether or not a country has double tax treaty agreements without considering tax information exchange agreements as a valid alternative for countries without direct taxation.


So TJN misses on this big one…


2).  How does the study control for other variables that affect financial size?


It DOES NOT. Essentially, if a given area has $30 billion in financial services, it has $30 billion in naughty financial services.  TJN’s final financial secrecy index score is the product of two functions of their secrecy score using their 15 indicators and the weighted financial size of each jurisdiction.  Holding everything constant, a country can effectively increase its score by providing $1 more in financial services.  Perhaps not surprisingly, seven of the eight largest financial centers occupy the top eight spots in the total score that combines secrecy and size.  Amongst these seven countries, not one achieves a score for secrecy that for this variable alone would place it in the top 20.


TJN’s argument for using financial size is simple.  A country with $30 billion in financial flows has a larger potential for illicit activity than a puny country with $1 million.  As such, it is understandable that TJN wants to control for size.  The only problem is, they don’t!  In the current example, the inclusion of any number of other factors, exchange rates, interest rates, and even the distance between importing and exporting countries, may reveal that for our first country all $30 billion of the financial flows were devoid of any nefarious intentions, with all $1 million for our second country attributable to tax avoidance by drug smugglers with bad credit.  In fact, while obviously not indicating a direction of causality, it is interesting to note financial size is negatively correlated with the financial secrecy variable in TJN’s sample.  From a pure correlative perspective, countries that are larger in financial size tend to be less secretive using TJN’s evidence.


TJN’s scorecard:  0 for 2.


3).  Are TJN’s results robust to modest changes in potentially tenuous assumptions?


The answer is a resounding no.  One having a relatively high threshold for pain might read through TJN’s “over 90 pages”related to their methodology to ultimately discover that the final financial secrecy index score is the product of their secrecy score raised to some power, call it “a,”multiplied by the financial weight raised to the power “1/a”.   We could stop here, catch out breath, and stop laughing, so as to point out that absolutely no economic or statistical theory is used to produce this hocus-pocus, or that the variable “a”is also picked out of thin air (TJN considers 1-5, but settles on 3).  But, let’s move on from this nasty scene.  Even if one could somehow force down this untenable assumed functional form, you would not be able to accept the fact that a change in the seemingly innocuous parameter “a”radically changes the final rankings.  For one considered value, the US dominates all other countries, ranking a cool 61 spots above Vanuatu.  A small change in the value of “a”from 1 to 5, pushes the US to 20th place and sees Vanuatu leapfrog to the 18th spot.


The fact that such a small change could so radically affect the rankings shows that TJN’s results are highly dubious at best. The study disguises itself as “high science,”which may even work to the passerby that doesn’t want to grab a shovel to dig through this stinky mess. As such, TJN’s rankings appear to be a list of large financial centers with superfluous random noise.


So, on our final question, strike three.  Careful analysis of the study revels serious methodological flaws that make the rankings uninformative.


The interested reader is invited to view the dirty details at TJN’s website, (  Underneath the heading, “if you disagree with us”(we do), one finds TJN’s defense of its methodology. There it reports, “we have applied our methodology consistently.” Even if true, who cares if it was done consistently, if it was done consistently wrong? One could have put all of the names of the various jurisdictions in a hat and assigned rankings on the basis of the order in which the names were pulled out. This is a consistent approach as well. It might even prove to be more accurate than TJN’s, but it is clearly not an appropriate way to select secrecy rankings.


For the full Study Click here

Support Terms of Use Privacy Policy