Experimental Economics in Social Sciences by Prof. Daniel Zizzo (University of Newcastle)

I’m a behavioural experimental economist by training and by profession. And what I’m going to do is to provide you with a flavour of what experimental economics is. I believe a lot in learning by doing, so the key way in which I’m going to do this is by running a little classroom experiment with a couple of simplifications just to save time. I will then show examples of how experimental economics can be of interest to CREATe. There will just be examples, I’ll go into detail more or less depending on how much time we have but I’m very happy to talk for the rest of the day or indeed after the conference to anyone who is interested. And I will finish off with just a taster on strengths and limitations of experimental methodology and of course it’s only one method. I’m not in any sense claiming this is the method, just one method to be used together with other methods and in conjunction with other methods.

So what I’m going to do is to ask you to pick up your pieces of paper and we’re going to play the following little game. All of you will guess a number between zero and one hundred okay? And you will write this on your piece of paper. Don’t write it yet, let me tell you what the rules of the game are. If this were a real experiment I would be collecting of all the slips, but there isn’t the time then for me to analyse all of them. I’ll just collect five at random. And then I will take the median value of them, for those of you that don’t know what median means, that means the number in the middle. And then I will take the eighty percent of this number. So I will take this median value, median guess and I will take eighty percent of it, and whoever among those selection that I’ve collected, whoever has guessed closest to eighty percent of this median value will earn five pounds. I’m saying I’m going to do it and I will do, it’s experimental economics, we don’t have deceptions but we do use money. So the prize is over here.

So repeat the rules, you need to think that all of you are trying right now to get to eighty percent of the median value of all the guesses. So you’re all putting some number, take eighty percent of that and if you can get closest to that you are going to get five pounds. Incidentally, were there to be a tie, I hope not, we’ll just throw it at random the prize among those you have tied, but I hope that doesn’t happen. Are the rules clear? So eighty percent of the median value and you need to write down a number between zero and one hundred. Please do not talk to other people, just keep it individual. In a real experiment typically you would be in your own little computer cubicle, you wouldn’t actually be talking to other people, unless the experimenter wants you to.

So take your time to write your guess. Everyone done? Okay another moment… Okay now I will just collect five. I’m going to take a moment to compute the winning number and then whoever is the winner raise their hand and we will have a big applause and give the five pounds. So the numbers I see here are zero, five, thirty seven, twenty four and thirty one. So the median value is twenty four. Eighty five percent of that is going to be somewhere between nineteen and twenty, so the winner is whoever has guessed twenty four. A big applause for the winner!

Good. Now in a real experiment unless I had wanted it to happen, I would not have had any applause, but that potentially is something else I am putting in so in this context that’s fine. I decided that I only picked five pieces of paper – in many real world experiments you would have choices and computer programmes. Now this is an example of a classic game called beauty contest game and if everyone here were thinking in what is commonly called a perfectly rational way, in other words you think about what other people do, but you also think that others think about what you do and what others do and you also think what others think what you think about what others do. And you also think what they think, what you think, what they think, what you think what they think about what others do and so on and so forth. You can see that actually whoever has guessed zero is actually straight there on the mark and that would technically be the winner.

But is zero the winner? No. Is zero the one that got the last laugh? No, not in this case. So if actually you know and realise that some people are not going to go down that far, then actually the median is not going to be zero, it’s going to be a higher value and therefore then you’re thinking mmm, if you were an average people are going to say 30, all I have to do it just to put something a little below that and that’s actually what’s going to get me the prize. So actually then this mark has moved while taking account in the best way, in the most correct way where you think people are going to stop. I mean if no one were to think about anyone else, then you would just say oh well, people are just going to put a number at random between zero and one hundred, they’re going to say fifty, I need to put eighty percent of that, so I will have to say it’s 40. If you’re someone who is able to realise that someone else will think but not go further then you will think well, someone else would be forty, so I have to put eighty percent of forty so what I’m going to do is say something like 32. Now if someone else is able to get to this second step then they will think about 32 and eighty percent of 32. This is why then the process unravels to zero because this process can continue ad infinitum, to infinity up to the point zero unless actually you think that people are not going to go that far, and if you don’t then you get your twenty four and win. Is that clear? Good, excellent.

So note some features of the task that we’ve played. You have made some actual choices and I haven’t asked you, I mean I could do this in experiments they can ask after a task, but if I ask you the sort of thing, oh well what do you think about X, oh well what is that you have done in the last year, and actually I’ve asked you to make an actual choice – an actual behavioural choice. This can have an impact on your pocket. And I had the effect – I transferred money from my pocket to the winner’s pocket. What you see is what you get, that’s a key difference between us and social psychologists who run, some social psychologists run experiments, I’m stressing some, I know that Piers my collaborator doesn’t do deception most of the time, just when he works with me. So we build up a reputation, people know that what we say is actually what happens.

The task is a very important feature in individual choice experiments, but in the task we played what I did, what you did depended on what other people did, there was this independence, right? So maybe you could think of it the choice to download will depend on what other people do in terms of downloading. The choice of sharing the file may depend on what other people do in terms of sharing and so on and so forth. There are interdependencies. Game theory and behavioural economics enable us to analyse these interdependencies. Experimental economics enables you then to study this and try to get some data out and make sense of it.

This is not a game that is inspired in any way by the creative industries. I guess the closest you could get is in thinking about a creative entrepreneur, someone in the fashion industry, you do wish to know what the fashion would look like in order to outsmart it, but you do need to know where is that it’s coming, enough beyond the edge so that you look the person who is fashion leader but at the same time not too far as to look totally bonkers, right?

So the degree to which people believe that other people are able to engage in behaviour which conforms to their self-interest in a logical way following all the way to the logic does matter. Sometimes you actually can make money out of knowing what the limitations of others are. But also social norms matter, either because you may wish to conform to them or because you may wish to deviate from them. So in the game which we did together, there was a bit of both, you both needed to think about me, you wanted to conform to some extent and also you wanted to deviate to an extent.

In other contexts, not this one, but in other contexts also you may think that other types of social preferences may matter. It may matter to the extent to which I may want to be reciprocal towards others, if someone is nice to me I may want to be nice to them for instance. It may matter the extent to which I care about fairness towards others in a number of ways, or it may matter the extent to which I want to be relatively better off than other people for instance.
So that has given you just a flavour of what experimental economics is about.

What I want to do now is just to present in a nutshell a very, very stylised crowd sharing game. Very stylised, very simple, too simple but something to use as a starting point and nothing more than a starting point. And because I want to start very simple, I will just start from two players, from two people playing but of course in the real world there are millions of players who are playing this game at the same time.

I’m going to assume that each of these players actually cares about the product, they add a value X to the product, X amount of money, we don’t know how much money, which is more than the price of purchasing. You could relax with this assumption I don’t think this is crucial in any way but I’m just going to do things in a simple way. I’m just going to assume that you like this and this. I’m also going to assume that there is a chance, a probability that if you share the product, so if you buy the product and you share it, you upload and this is found out you have to pay a fine. Of course this is stylised, you could get a nasty letter for example, you could have your internet rights revoked, you could have all kinds of things. Equally I’m assuming that if you download the file there is a probability that they catch you out and that there is an undesirable outcome of some sort, if you believe there are none there could always be some zeros, it doesn’t matter. So then there are four possible actions that the two players can take:

i) They can buy and not share the file
ii) They can buy and share the file, upload it,
iii) They cannot buy and download, of course this does assume that the other player shares because otherwise you can’t download it without being uploaded in the first place, and
iv) Not buy and not download.

Now if these players (and this is a very, very big assumption, too big but it’s just an assumption), if the players are rational and purely self interested, they don’t care about anything else but themselves and what they get from music then it’s straight forward to see that there is a paradox of sharing and that’s because the only effect of sharing is to have a risk that you’re going to be caught, that’s the only thing extra that you’re going to get if you’re rational self interest. So under the assumption of rational self interest it simply doesn’t make sense, the paradox is in the share. Conversely downloading is absolutely no problem, if you are of rational self interest and as long as there isn’t a big risk of getting caught, as long as you don’t extremely dislike a risk you should just download, it doesn’t make sense to do otherwise. Clearly though the economic parameters, the probability both of being detected and being fined may make a difference.

Now in a behavioural model, you would introduce additional preference dimensions that would explain why sharing occurs in the first place. Reciprocity, the fact that I care about what you think of me, what my peers think of me, and if I care about what my peers think of me then I may be willing to share because then they’re going to like me and they’re going to show that I’m cool, that’s what I’m going to do and then that may enforce a sort of norm of sharing or downloading if necessary. Of course though you may introduce deviations from rationality or indeed from self-interest, even small deviations may make a difference and this is particularly relevant if there are loads of players because think about it, if there are millions and millions of players rather than two then all it takes is one person who uploads the file maybe because ideologically they believe that file sharing is great and good and to share knowledge, so for ideological reason you think that’s the right thing to do, you only need one person and then everyone else could download.

What you immediately see already from a simple model like this is actually you may have a situation where sharing and downloading may be explained in very different ways and the typology of who shares may be a very peculiar typology that concerns only point zero one percent of the population and still it would explain everything; once you have got that everything else follows. And I think some of the preliminary evidence that we have got from the scoping review is along these lines. Of course there may be additional dimensions and I think precisely what I’ve stated, there may be differences between downloading and sharing.

So a basic economic experiment on file sharing, very basic, too basic but we need to start from something simple and move from there would implement the file sharing game in one way or another. We would assign a monetary value ten pounds say, to the utility someone would get from the file which is a standard way of proceeding in experimental methodology, one of the Nobel Prizes, Vernon Smith, for economics that he got the Nobel Prize for his work in economics it was part of this, you assign monetary values to the utility that people get from commodities and the personal benefits that people get from commodities. And it does have an advantage of allowing perfect control and preferences but of course if I replace a real file with a piece of money there is a question about is talking of monetary values the same as talking of pleasure of listening to music, so there are trade-offs.

You could of course use a frame in the game in the experimental set up in terms of file sharing, that’s something you could do. You could frame everything in terms of file sharing in order to make it a bit more closer to the real world. And you could select and vary the parameters ever seen. Indeed you could have treatment manipulations that try to isolate the psychological factors. So you could for example consider variations in economic parameters, you could consider automated players that would make particulars choices and see how others react. You could manipulate the number of players of this treatment because we saw that this actually made a difference. Of course where there are differences in terms of across subjects, whether if you have psychological measures there are important relations. For example this is something that Piers and I are quite interested in; how much you are sensitive to social desirability effects and in this context sensitivity could be interpreted for example could be a sensitivity to social pressure by your peers, but it also could potentially be sensitivity to receiving a nasty letter telling you you have been downloading and that’s bad. And you could introduce all kinds of rewards and punishments in different treatments.

And the key of this in experimental manipulation is that you would have precisely the same game, precisely the same environment but you would change with different people, you would change one parameter, you would change one thing so that if then you have a difference in behaviour you would be able to causally attribute that difference to that one thing changing between the two treatments. That’s the advantage of experimental methodology.

Another area that I’m interested in for the future, this is not my work package I should say, but it does interest potentially quite a few people in the audience and it may interest me in the future as well is our questions related to the fostering creativity, so one could look at how that [creativity] interacts with economic incentives. There are a couple of excellent papers in economics that have looked at that. Later today I think there will be a discussion on papers by Buccafusco and Sprigman about the value of products being created and indeed is there something special about creatives, is there something really special about the people who create. I know that Ruth for example is interested in this.

So what the experiment will do is to allow you to collect data, to investigate factors which otherwise you would find difficult to do. The virtue of experiments is that you control the decision making environment and are able to vary one factor keeping everything else constant. That is what allows good influence on causalities.

Now there are different ways of doing this, I’m not going to go onto the detail for reasons of time but you could do this either with the same people playing the game in the same sequence for example in a slightly different way, the one treatment manipulation, the one thing that changes but the same subjects plays both treatments, that’s both manipulations or you could have that older people to the right do the experiment in one way with one manipulation for example eighty percent of the median whereas everyone on the left would play the same game as we played at the beginning but slightly different, for example one hundred and twenty percent of the median rather than eighty percent of the median and then you would be able to compare, to make that comparison between subjects. And there are advantages and disadvantages of both ways of approaching things.

What experimental methods are particularly strong on and what psychologists label as internal validity so is an ability to know if there is a clearly designed experiment that did know precisely what is going on in the experimental lab. Why what is happening is happening is happening. There is however another dimension and that is the dimension of external validity and that is the ability to generalise from the research context of the settings that the research has been an approximation. In other words that’s what happened in the lab generalised to the natural world. I’m not going to say real world particularly because what happens in the lab is also the real world and it does happen there, it’s concrete, it’s not in metaphysics, but the natural world. And what is normally said is that experimental data is comparatively strong and internal validity and comparatively weak and external validity relative to field data. So it’s one methodology that’s got strengths and it’s got limitations.

So then how can one face this challenge, how can one improve issues of external validity? How can one improve what you can learn from an experiment of the laboratory? Or what kind of defences one can make against arguments based on the fact that oh it is just university students doing that stuff in the lab that doesn’t really tell me anything about the real world. This is a problem of external validity, in the real world, if you want to talk about real world most people do, that’s the question. What does that experiment tell you about what happens in Edinburgh?

Now the more constructive way of rephrasing this question is the following: If you think that what happens in the lab is different to what happens in the real world then you should also say, what is it that in your view in any particular case, what makes the experimental lab different from the real world? So it’s not enough to say oh well clearly it doesn’t make any sense, but you also need to tell me what is it that you think makes a difference. If you just tell me oh well some things are really different, this is basically just an argument….I mean you could tell this based on every kind of data, it’s not something specific to experimental data, it goes back to [name 01:35:55] thesis on the limitations of induction, about the impossibility of supporting or falsifying a theory because you can always support, revise it and then support it with suitable abstract hypothesis. So virtual, empirical, legalism. So that’s why what you need are specific climates.

Now there are a number of common criticisms and if we had the time which we don’t I would go over each of these, and if anyone’s interested I’m happy to discuss them. Things like ‘you use university students’, ‘the task is artificial’, ‘there is too little money’, ‘you don’t allow people to learn’. These are all criticisms that can be addressed with suitable modifications.

So you could use field data to compliment experimental data, this is one possible strategy to answer this type of criticism. Or you could run field experiments and I’ve done a couple of field experiments myself so for example I’ve done now an experiment with competition policy practitioners from real competition authorities from all over the world as opposed to university students and since this was an experiment on competition policies you were making that was a proper sample. Interestingly you actually got similar patterns of information just by using university students.

Or you could run additional experiments in the lab in some cases to control other factors. A second experiment, a third experiment and so on. In principle the list could be infinite but in practice you could think of a list of possible factors that may be found, and there are debates within experimental economics that have kind of reached some kind of agreement.

So let me conclude, experimental economics is one method of collecting data. I repeat it’s one method, it’s not the method but it is one possible method to consider perhaps in conjunction with others. It often uses also insights and perspectives from behavioural economics which very often uses ideas and insights from its cousin experimental psychology and social psychologists. And in the realm of CREATe it has been used to a limited extent so I see great opportunities potentially out there to explore. Thank you very much.

Comments from the floor

Prof. Derek MacAuley (Nottingham): So I have very briefly a two part question for the audience and then while you’re thinking about your answer, so who else in that task chose a transcendental number? Okay. I’m the outlier. Who and I want a show of hands, knows what a transcendental number is. Three. Okay. I am that outlier. I’m probably the guy who does do the file sharing and messes with the system because that was my first take on this. Questions?

Question 1: What’s a transcendental number?

DM: A number that’s not a solution to an algebraic equation. A polynomial. Bubbles you say.

LE: I probably should know the answer to this by I’m still getting over the jet lag. The original assumption that you value the file as much or more than the selling cost seemed to me to be wrong for a large percentage of file sharers who download speculatively for something that they want to try before they buy and therefore they may value it not very much or indeed not at all, they may end up hating it. And I just wondered what the implications for the file sharing game of that were because I think the whole chain of reasoning you then followed derived from the original set condition that you did value the file at least as much or more than the paying price.

DZ: Actually I don’t think it does because you see the problem here, what you have suggested is something that is relevant in the context of the downloader, because for the sharer in order to share you need to have the file in the first place, so someone needs to upload it at some point. From the perspective of the downloader, well it doesn’t really make much of a difference because anyway according to self interest you would download anyway so it doesn’t make any difference. All that matters as a downloader is that you have a positive value to the file, anything else doesn’t matter.

Question: I have a resource question, so I’m quite excited about the possibilities of experimental economics to answer some of these questions, but in terms of the time and the cost to run some of these experiments, how does it compare to other methods? Fixed and variable costs associated with it?

DZ: Okay a typical experiment would be done with computers, not all of them, I myself still do experiments with paper and pencils that are experiments for scientific journals. So that is possible and if you don’t have partitions, when I was a Masters student I used cardboard partitions it’s very simple, it only requires very limited resources. You would still need to have the money to pay for the subjects, that depends on the scale of the experiment, how ambitious it is. If the experiment requires just choices in isolation, not interacting with other people you can probably do it one the cheap even for five hundred pounds, fifty subjects, that would be the minimum I would say. If it is more complex, you will need more money. If it is computerised you will need a place to run it, but for example I’m aware that Edinburgh just last weekend opened up their new laboratory. There are laboratories out there, East Anglia is one but it’s not the only one. Nottingham has got a great lab indeed East Anglia is connected with Nottingham so there are possibilities out there to explore as well, does that answer your question?

DM: Daniel thank you very much that’s been a second illuminating talk this morning, it’s been a buzz. Let’s thank Daniel again.

Prof. Martin Kretschmer (Glasgow): Sometimes it looks to me like the internet is a big experimental machine because almost all activities leave a trace and if any setting is changed, the whole way Google operates, or Facebook operates, the change in behaviour could be treated as a field experiment. So I don’t know whether that’s true, but I’m inclined to that view of data protection on the internet.