Hypothesis testing

Hypoth­e­sis test­ing is, accord­ing to my opin­ion, anal­o­gous to the sci­en­tif­ic method. It fol­lows a log­i­cal struc­ture that enables an objec­tive pro­ce­dure that allows sci­ence to progress. Hypoth­e­sis test­ing is an essen­tial aspect when it comes to the plan­ning, exe­cu­tion, analy­sis and inter­pre­ta­tion of results of a research project.

The first step in this pro­ce­dure involves the con­struc­tion of a null-hypoth­e­sis (H0). This is the oppo­site of the researcher’s hypoth­e­sis (H1), which actu­al­ly should rep­re­sent a the­o­ry that may explain a spe­cif­ic obser­va­tion. The alter­na­tive hypoth­e­sis can be viewed as a pre­dic­tion of what will occur if the the­o­ry is cor­rect. It is the null-hypoth­e­sis that is to be test­ed with a sta­tis­ti­cal test. The rea­son for this relies on philo­soph­i­cal grounds; Pop­pers fal­si­fi­ca­tion­ism. In short, this con­cept says that it is not pos­si­ble to say that any­thing is true unless you have gath­ered all pos­si­ble obser­va­tions, which in prac­tice is impos­si­ble. But, it only requires one obser­va­tion to fal­si­fy a hypoth­e­sis. If a hypoth­e­sis once has been fal­si­fied, it remains false. Then it can­not be true. For exam­ple, you have a belief that all flow­ers on the plan­et are red. Every day you only see red flow­ers, so this is your “truth”. But one day you go out for a stroll beyond the lim­its of your own gar­den. Then sud­den­ly, you see a blue flower. Your belief is false. It only required one blue flower. Your hypoth­e­sis is only true until the day it is fal­si­fied. This means you can nev­er say that some­thing is true. Once the null hypoth­e­sis is fal­si­fied you receive sup­port for the oppo­site; your alter­na­tive hypoth­e­sis, H1. This is only sup­port, not the truth. It is “true” to the day an exper­i­ment is unable to fal­si­fy the null-hypoth­e­sis. Then the alter­na­tive hypoth­e­sis have been fal­si­fied.

The null-hypoth­e­sis is often set to zero as in no effect. But you can also see it as a base­line from which your alter­na­tive hypoth­e­sis devi­ates. In this case, the null-hypoth­e­sis does not have to be zero. The null- and alter­na­tive hypoth­e­sis can be expressed in words but also in more math­e­mat­i­cal terms. For instance, say that your alter­na­tive hypoth­e­sis is that “peo­ple are taller in Swe­den com­pared to Japan”. The null-hypoth­e­sis is then: “Peo­ple are as tall or taller in Japan com­pared to Swe­den”. In math­e­mat­i­cal terms the alter­na­tive and null-hypoth­e­sis can be expressed as:

H1: µSWEDEN > µJAPAN

 H0: µSWEDEN ≤ µJAPAN

Where µ rep­re­sents the pop­u­la­tion mean of heights in each coun­try.

The sec­ond step is to deter­mine which kind of exper­i­ment you have to do or which kind of data you need to test the null-hypoth­e­sis. In this phase, the con­cept of sam­pling is impor­tant in order to get a rep­re­sen­ta­tive sub­set from the pop­u­la­tion in ques­tion. Before the sam­pling in com­menced, you should invest time to deter­mine which kind of sta­tis­ti­cal test you need to per­form, which lev­el of sig­nif­i­cance you should use and which sam­ple size you need in order to detect a dif­fer­ence if it is there. The lat­ter has to do with the pow­er of the test.

The third step involves the exe­cu­tion of the sta­tis­ti­cal test of the null-hypoth­e­sis.

In the fourth step, you inter­pret the results of the test, which should not be very dif­fi­cult if you have spec­i­fied an alter­na­tive and null-hypoth­e­sis in advance. For exam­ple, you have sam­pled the Swe­den and Japan pop­u­la­tion of heights, per­formed a sta­tis­ti­cal test (in this case a t test) where the mean height in Swe­den was sta­tis­ti­cal­ly sig­nif­i­cant larg­er com­pared to the mean height of Japan (µSWEDEN > µJAPAN). This means that the null-hypoth­e­sis can be fal­si­fied and we get sup­port for the alter­na­tive hypoth­e­sis and can con­clude for the moment that peo­ple are taller in Swe­den. If the hypoth­e­sis is a pre­dic­tion of a the­o­ry the inter­pre­ta­tion can go fur­ther, claim­ing sup­port for the the­o­ry.

The fifth step is in sci­en­tif­ic and philo­soph­i­cal terms very impor­tant but can­not always be done because of prac­ti­cal issues. It is the con­tin­u­ous work of try­ing to fal­si­fy the alter­na­tive hypoth­e­sis. We only receive sup­port for it when the null-hypoth­e­sis has been fal­si­fied. It is not the truth.

My expe­ri­ence is that you have to be some­what prag­mat­ic about the pro­ce­dure of hypoth­e­sis test­ing. It is not always pos­si­ble to spec­i­fy hypothe­ses based on a the­o­ry. This requires hard work con­duct­ing descrip­tive stud­ies from which obser­va­tions are made that can be explained but some the­o­ry. The descrip­tive stud­ies, how­ev­er, involves sta­tis­ti­cal tests in order to pro­duce pat­terns. Then by rou­tine the null-hypoth­e­sis is set to zero or no effect/no dif­fer­ence. The key word in the process of using hypothe­ses as pre­dic­tions of a the­o­ry is a pri­ori work. That means, you need to know in advance what you are after and plan your project accord­ing­ly. Oth­er­wise you are wast­ing your time.

Sum­ma­ry

Con­struct an alter­na­tive (H1) and null-hypoth­e­sis (H0)

Plan the experiment/sampling and decide which test to use

Per­form the sta­tis­ti­cal test

Inter­pret the results of the test

Con­tin­ue your research try­ing to fal­si­fy your alter­na­tive hypothe­ses that sup­port your the­o­ry