Willem Janssen [00:00:00]

Welcome to Bestek, the Public Procurement Podcast. Today we’ll be talking about award criteria, putting them under the magnifying glass.

About Bestek [00:00:13]

Welcome to Bestek, the public procurement podcast. In this podcast, Dr. Willem Janssen and Dr. Marta Andhov discuss public procurement law issues, the love of food and academic life. In each episode, William, Marta, and their guests search for answers to intriguing public procurement questions. This is Bestek. Let’s dish public procurement law.

Willem Janssen [00:00:38]

Hello. We’re back.

Marta Andhov [00:00:39]

We’re back and we’re recording again on site in Copenhagen.

Willem Janssen [00:00:43]

Still at the faculty of Law in Copenhagen. And we’re joined today by Rasmus Horskjær Nielsen. Did I do that correctly?

Rasmus Horskjær Nielsen [00:00:50]

Yeah, it was quite good.

Willem Janssen [00:00:51]

Quite good. I’ll try better next time. Maybe at the end I’ll try again.

Marta Andhov [00:00:56]

More practise.

Willem Janssen [00:00:56]

But it’s a pleasure to have you here today in Bestek, the Public Procurement Podcast, and we invited you here today to talk a bit about your PhD, which you recently defended about award criteria. And for the for the listeners, you’ve done basically every job there is to do when it comes to public procurement law or competition, I think, or at least from a lot of angles. You’ve worked at a big law firm as a lawyer. You were an academic. You were counsel at the Danish Competition and Consumer Authority, and now you’re an in-house lawyer at Andel, which is a public utilities company. So, I feel like all of those aspects will have come back in the PhD at a point, or at least all those perspectives. But clearly it was an academic one in the end. How’s life been since since you finished your your Ph.D.?

Rasmus Horskjær Nielsen [00:01:47]

Yeah, it’s been quite good. I’ve been in academia, maybe the shortest career in academia ever, three months. But then I changed to Andel, as mentioned. And yeah, it’s been quite good since then.

Willem Janssen [00:02:02]

So, I think you’re being too modest, right, too. You did the PhD, which I don’t think you wrote in three months, but it was the position afterwards. Right. A couple of years. So, the perhaps it wasn’t the shortest academic career because some people leave straight after their PhD.

Marta Andhov [00:02:15]

Well, and let’s also hope that’s not that we did such a bad impression on Rasmus that he just run away.

Rasmus Horskjær Nielsen [00:02:22]

 No, no, it was a very good impression. But, yeah, I got a good job oppurtunity.

Marta Andhov [00:02:27]

It’s one of those too good to pass on, right?

Willem Janssen [00:02:32]

So let’s scope into your Ph.D. What we’ll be doing today is, I think, talking about some of the main points that you’ve tried to argue for in that research project. You have it lying in front of you here. It’s a big book in printed in a A4 and we’ll be highlighting some of those aspects. But before we do that and before we also go to main and to dessert, well, where hopefully you’ll be giving some out, some advice about what to do and perhaps what not to do. What what is the advice in hindsight that you can give to PhDs. Let’s, you know, set the scene a bit. What are we talking about today? The focus is award criteria. But let’s start with why they are so important and why should we actually be talking about them in this podcast.

Marta Andhov [00:03:19]

And why actually you chose that topic? What got you to actually decide? Yeah, that’s something that I can work for three years on.

Rasmus Horskjær Nielsen [00:03:26]

 Yeah. A good question. After you’ve been working with that topic for that long, you also think, oh, why did I actually choose it, but yeah, but in my opinion, competition is at the core of the public procurement process. So even though we have a lot of different rules, well, the competition is in the core of the procurement process because we need to have some kind of a bidder we want to go into contract with. So, and when you look at the public procurement directive, actually there are very few rules regulating the competition. There’s Article 67 and there are a lot of different rules here and there. But at the end of the day, the competition is very, very regulated and very few articles. So, so I looked into two competition and there are a lot of problems when you have a public procurement process, a lot of the problems you encounter regularly are related to the specific competition. So how are you going to set the award criteria and what is the weighting and how are you going to identify the most economically and advances tender. So, it’s very difficult and how do you use the evaluation methods as a mechanics to reach this goal. So, there was a lot of problems from a legal perspective and I thought that that would make a great topic for an PhD thesis. So, I went about to look at the topic. And at the start I saw that there were you could say two topics. You had the the first topic where you have to set the awards criteria, the weighting and the evaluation method.

So, what needs to be in the public procurement or the procurement documents. And then you have the second part of it is when you evaluate tenders. And at a starting point, I thought that my thesis would cover both those topics. But quickly I realised that that would be …

Marta Andhov [00:05:54]

Three PhDs.

Rasmus Horskjær Nielsen [00:05:55]

 Yeah, exactly. So, I narrowed down to just the creature weighting and evalutation method. When you have to set them in the procurement documents, what rules and boundaries are there. Snd that is, the main topic for my PhD.

Willem Janssen [00:06:12]

That’s fantastic in a way to start with, with frustration and difficulties in practise. Right. Because generally I also find that they’re rooted in more fundamental problems that are interesting from an academic perspective. So, I think why it’s so important to keep in touch also with when I speak to, to give you an example, when I speak to contracting authorities in the Netherlands, I would say that evaluation is by far the one of the most difficult aspects of the public procurement process because in a way, choosing for a certain type of procedure, okay, manageable to set certain technical specifications if you’re aware of what the market does and what you would want in terms of perhaps sustainability level, also difficult but manageable. But when you actually need to start thinking about how can these firms actually or these economic operators compete and how can we manage to get the most out of what they’re willing to offer, that’s all of a sudden, a whole different ballgame. So that’s why I was also happy to hear that you looked into this, this aspect. And I suppose this when you say that would have been three books, there’s still time to write the other two, right? So, let’s hope they actually get written at a point.

Marta Andhov [00:07:19]

And I think it’s also a very good point as a point of reference or advice for anyone that is doing a PhD or is considering doing a PhD is just that the work that Rasmus did and we right now is, as Willem mentioned, we looking at the PhD in the way how we had a chance to look at a defence. But, since then Rossum’s also published it, unfortunately or fortunately, because this is when we become very unique because you can hear about the thesis in English for now the thesis is only in Danish, but I know that I think Rasmus at some point was considering writing something in English, so there might be something coming at some point. But until then, the podcast will be the thing to look at. But it’s also extremely practical, right? That means that whatever the work that Rasmus did with the PhD potentially can really help out someone in practise. And I think that’s also a great part about our work

Willem Janssen [00:08:14]

For sure. So, we’re hoping that, you know, whatever insights or how nerdy they might the discussions might be today that they actually would help people or or, on the other hand, spur the academic debate when it comes to this. So, when we look at this topic, right, you introduced it. We’re looking at Article 67. And if you would include lifecycle costing, perhaps we can include 68 as well. But like the focus is 67. And in the classic directive, when we talk about the status quo of the of the law, where are we at right before we start tearing the house down? What are we actually what’s the house? And because you mentioned it yourself as well, and that’s been the core of your thesis when it comes to transparency in the announcement, like what do we have to announce and what can we announce? What where’s the distinction made in terms of award criteria?

Rasmus Horskjær Nielsen [00:09:02]

Yeah, the status quo is that you need to publish all criteria and you need to publish the weighting of the criteria, at least at a subcultural level. And then you have the, uh, that is, that is like the publication requirement, but then you also have a requirement that the award criteria, they must be linked to the subject matter of the contract. They can’t give the construction authority an unrestricted freedom of choice. You also have the requirement that it should be able or suitable to identify the most economically advantageous tender. So, there are a lot of the requirements when you set the war criteria and then you have to publish it. But in my thesis, I go into, so we have and from the directive and from court cases we have like a list of requirements. But I go into my thesis and look at what is what does this these requirements actually mean? What can we do and what can’t we do? And for instance, a question is, is there an obligation to to publish the evaluation method? And is there a possibility to have crisis subject to award criteria, or should it all be measurable? So, there are a lot of even though that the requirements are quite fixed, there are a lot of discussion on how we understand these requirements. And I totally agree with Marta that this is very relevant in practise. And that is the perfect thing with public procurement law. We have an academic view, but it is all very relevant in practise and regarding competition, I hope that my conclusions and my thesis that they can be used directly or at least give some pointers to a discussion.

Willem Janssen [00:11:14]

It’s nice to talk to a public procurement aficionado. We also speak to Ole, a contract law specialist academic here at the university. He was a bit more hesitant; I think.

Marta Andhov [00:11:25]

He’s not buying into the Coolio.

Willem Janssen [00:11:28]

So, I feel like we found another public procurement. Well, can I call you a nerd as well?

Rasmus Horskjær Nielsen [00:11:33]

Yes, yes, yes, please.

Willem Janssen [00:11:35]

I say that with the highest regard, of course.

Marta Andhov [00:11:37]

And there is also probably just to add to what Rasmus already described, there is also slight differentiation that also makes it interesting in practise between member states because in context of the evaluation methods ,and we for sure will hear about it from Rasmus a little bit more, but the European level, more or less, tells us you don’t have to always, under all every circumstances, publish the evaluation method. But if we look, for example, with the Danish system, with the Section 160, Rasmus, correct me if I’m saying something wrong. We have the requirement of publication of evaluation method. So I’m wondering that probably a variation of similarities and differences in approaches across Europe will be existing, right?

Willem Janssen [00:12:27]

For sure. Because also just to quickly add to that, in the Netherlands, it’s not obligatory to publish the scoring method or the evaluation model, whatever you call it. It’s still up to the discretion of the contracting authority. Some do it, most of them don’t. That’s, I think, the status quo here in the Netherlands. So, I think that’s why it’s also an interesting topic to to to get into. So, we have some regulation on the EU level. Some member states have regulated a bit more like in Denmark, there’s a difference between what needs to be published and what doesn’t need to be be published in the announcement of the tender or in the tendering documents. And with that recap in mind, let’s let’s dive in a bit.

Marta Andhov [00:13:12]

Let’s let’s go to our main.

Willem Janssen [00:13:14]

Yes, exactly.

Marta Andhov [00:13:15]

So, Rasmus, if you can introduce this more broadly and in the second, we will get into this more technical detail kind of nerdy stuff, but if we would to try to paint a picture for listener or future reader of your thesis, what are the main takeaways? What is the main sort of points that you focus on that we will dive into a little bit more in this discussion?

Rasmus Horskjær Nielsen [00:13:41]

Yes, you could say if you look at my thesis or it wasn’t apparent for me at the beginning, but at the end I could see that we have a lot of court cases regarding the competition. But, they can be seen separately or we can see them in in in some kind of big storyline. So, as I also mentioned to to my P.S. this defence, there’s an old Chinese saying where you don’t look at the wave, look at the current, and that is what I’m trying to do in my thesis, look at the current of the different court cases. And in that view, I look into that there are two currents. You could say that you have a lot of requirements that ensure a relevant competition. So, you have to have transparency. You have to have a competition that is in line with what the directive says. So, an equal competition. And then you also have a competition where you are able to identify the economic most economic advanced tender. So, that is a lot of requirements to ensuring a competition for the bidders. Then you also have some requirements in relation to ensuring a control so that the bidders can control that the competition is in line with the principles and all the requirements. And if we look into that, the relevant competition, transparency is at the heart of that that current, you could say, because you need to get information as a bidder and you need to get information about the competition to evaluate if the contract is all of the procurement process is relevant for you. So, you need information about the award criteria, you need information about the weighting, but also you need the information at a point where you can assess the contracts. So, that’s a priority in the contract notice, and that information should be clear and stable throughout the process. Then you also have some kind of requirements for the specific award criteria. So, the criteria must be objective. They can’t allow an unrestricted freedom of choice for the construction authority and all that is regulating the competition. And in that view, you could also view a look at the evaluation method where we have the great question is the evaluation method, is that an information that is relevant for bidder as a part of the relevant competition? Is it that a relevant information or is it only a relevant regarding the control or is it or is it none of the above? And so, if we look at the main points of my thesis, I think that I tried to structure the different requirements within assessing of the court cases, and there are court cases that deviate. But I think that we in academia try to like make a path, look at the path we have travelled and say, well, we are going this way and I try to say that we are going towards or have travelled the way of relevant competition and control competition. But I don’t know if you agree with that.

Marta Andhov [00:17:18]

I think I think that definitely, as you might remember from the defence, there was some question around where you started, but definitely there was acknowledgement of a huge amount of value in, in the work that you’ve been doing, particularly the challenge of understanding and maybe that you can tell us a little bit more is because exactly if you read through the law, you won’t find a reference as such to would you describe relevant competition or controllable competition. Can you then tell us a little bit more about how you see those two concepts?

Rasmus Horskjær Nielsen [00:17:56]

Yeah, and I agree, there isn’t anything. In the procurement directive or in court cases where they talk about the relevant competition or control over competition. But when you look at the results from the court cases, it’s, it’s the different waves signifies the current. So, if you look at the relevant competition, well, you have to have a transparency in what is the competition, so that hat the bidders can evaluate if they want to participate and they can structure the bid. But if you have an information that isn’t relevant in that sense, that it is a relevant information for the bidder, well, then you don’t have to have it in the procurement documents. And that is not only relevant for the competition, it could also be relevant for other information, maybe information regarding the contract. We have seen with the Siemens case where you had to put in a maximum value because you need to know if you actually can fulfil the contract, even though that doesn’t really relates to the competition it’s the same line of thought that you need to have information that is relevant for the bidder.

Marta Andhov [00:19:10]

So by relevant competition in different way or wording it in a different way, you would say the type of information that is relevant for competitors to decide A if they want to participate in tender, and B, how that information impacts the way how they would design their offer.

Rasmus Horskjær Nielsen [00:19:31]

Yeah, that’s fair to say. And that’s just the transparency part of it, because a relevant competition, you could also call it a public procurement competition because it isn’t a free competition as in other parts of private law and so on. It’s a specific competition. And with the requirements I said transparency. You have to have the public procurement kind of competition and you have to have a competition where you can identify the economic most advanced stages tender. Then you have a specific kind of competition and all the requirements. They ensure that you have this kind of public procurement kind of competition, and there isn’t really a word for that. And then and that might have been a bad idea, but then I made up my own word and it’s relevant competition.

Marta Andhov [00:20:23]

You know, that’s the part of PhD, right? Creating concepts. So, I think that you have a good reason to do so. Okay. So, I think that we kind of have a grasp of what the relevant competition for the purpose of your thesis means. What about controllable competition?

Rasmus Horskjær Nielsen [00:20:39]

Yes. Controllable competition is the control that the bidders should have over the public procurement process. So, you have to have the ability to control if the contracting authority actually have fulfilled the rules they’ve set for themselves, but also have the principals. So, and in that way, transparency is also a part of controllable competition because even though you have information that are relevant for the bidders and then it must be a part of the procurement documents, it also allows the bidders to have a control afterwards. But you also have information that are not relevant for the specific competition or relevant for the bidders, but they can still have a relevance for the control afterwards. And then you have to have some kind of different system. So, you have like a public procurement, relevant competition. And then you have the possibility you must have the possibility to control if that competition is in line with the principles and the directive.

Marta Andhov [00:21:49]

So to give a simple example, when we talk about controllable element of the competition is the fact that I know you said that you assign, let’s say, 60% of award criteria to quality and there needs to be enough transparency in the process that if you suddenly change your mind and it’s 75%, I need to be able to find a buyer to challenge such a change and such a award. Is that fair to say?

Rasmus Horskjær Nielsen [00:22:14]

That’s fair to say is exactly the point that because if we don’t have a control, we don’t have the ability to see if the contracting authority actually have fulfilled their obligations. So, you need to have a system with control, and that can be transparency, but it can also be other requirements.

Marta Andhov [00:22:35]

So Rasmus, in your approach, I would say that you go very much with this theme of we need more control over ultimately what contracting authorities are doing. And you also make the statements in the line of argumentation, of pointing out towards the fact that we need more control over award criteria or award criteria need to be subjected to a higher degree of control. Why is that so and how we can do it?

Rasmus Horskjær Nielsen [00:23:06]

That’s a good question. We have a situation where contracting authorities, they feel they are getting attacked almost that way. You say, well, we need control. And the contracting authorities say, well, why don’t you trust us? But in my view, the public procurement directive is, for lack of a better word, it is a possibility to control the construction authority so that you could say that trust is good. Control is better.

Marta Andhov [00:23:34]

And just to jump into your word I think this is also how the ultimately the roots of how we design European public procurement. That’s the set up, isn’t that we said we actually don’t trust them that much. There’s some discretion, but we generally don’t trust the system that much. So those are checks and balances to ensure that. Right?

Rasmus Horskjær Nielsen [00:23:52]

Exactly. And if you have that notion, it makes the rules easier to understand. So, yeah, you could say that the procurement directive is like a miss. They don’t really trust the contracting authority, but yeah, that is the state of the law. So, and in that view, in my opinion, it doesn’t make really sense. We have all these rule rules relating to public procurement. If we at the end just can choose whoever we want because we have very soft metrics or soft criteria where we just can say, well, you get ten points and they get six and we can really control it. So, or verify it. So, in my opinion, you need it’s within the scope of the system that you need some kind of verification and control so that the procedure procurement process makes sense.

Willem Janssen [00:24:46]

Okay. So, taking that point, right, as a given, we need stronger control when it comes to award criteria. You could say, well, that’s not award based on sustainability criteria anymore. We don’t need further oversight because there’s less discretion involved. I would be interested to hear your thoughts on that and hopefully your answer will be no. We should still be more based on sustainability criteria.

Marta Andhov [00:25:11]

I also just want to interject that I reject the notion that sustainable criteria are not objective or measurable.

Willem Janssen [00:25:17]

Exactly. But that was the second part about setting it up. Thank you so much for that. Because I think that’s where it gets tricky. When you look at many countries that are still struggling with high levels of corruption or conflict of interest, they generally say, well, let’s go for lowest price because, you know, sustainability is too subjective. That will lead to more nepotism, more contracts being awarded to, you know, friends and family. What would be ways or what would be in that scenario when you talk about greater control ways and tools of how contracting authorities could, in line with the law, make sure that that objectivity is there in a subjective world.

Rasmus Horskjær Nielsen [00:26:02]

It is very difficult because I agree that there is discretion and that you can’t only have measurable criteria. You need to have the discretion and the possibility to take takes into account. And in my opinion, you can’t have very flexible, award criteria without the checks and balances. And that is the heart of the question. So, for instance, I have in my thesis an example of coffee.

Willem Janssen [00:26:36] A very good example.

Rasmus Horskjær Nielsen [00:26:41]

There are very different opinions about coffee, I’m quite new to the coffee scene. I was introduced to it in Italy, in Torino, and I can’t get enough of it, but I see that there are very different coffees

Marta Andhov [00:26:59] Strong opinion about good and bad coffee.

Willem Janssen [00:27:00]

. Let’s just say you’re talking to a double certified home barista who actually managed during COVID, took on a couple of online courses. So I know everything. No. I know a lot there is about coffee, but keep going. Don’t feel pressure but like, you know, who you talk.

Marta Andhov [00:27:14]

If we if we want to do coffee, how we do it the right way and how it’s done in the way that it’s questionable, then.

Rasmus Horskjær Nielsen [00:27:22]

Yeah. Then, you would agree that you can really measure the taste of coffee. You can have some kind of criteria where you look at what are you going to give a positive evaluation like is bitter and so on. But you can’t really put it into a folder. And in that case, I would say that you have to have some other checks and balances, for instance, that you don’t see who is the supplier when you taste the coffee, because you need to taste the coffee to know if it’s good. But if you don’t know who the supplier is, you create some kind of checks and balances where you ensure that there is no risk of favouritism.

Marta Andhov [00:28:01]

You see you’re trying to neutralise the potential issue. So, another aspect of it is in when you buy food and I know Copenhagen municipality, when they conduct the tenders on food, they also really stand by the notion of having the sort of tastemakers and the argument being it’s not actually anyone who is involved in the procurement evaluation that taste the coffee, but they actually get, you know, sort of cooks with qualification and years of experience that you can argue the objectivity of the fact that they are capable of trying something and they have refined tastes to actually be able to say, this is good. This is not in combination what you described. Resumes also the anonymity of the samples that you tasted that sort of together gives enough good of an argument saying that taste, which might be a subjective criteria, is carried out in the most neutral and objective way possible, right?

Rasmus Horskjær Nielsen [00:28:58]

Yeah, exactly. And I think that the way the Copenhagen municipality is going about it is the right way to ensure that the award of a contract is a result of a competition and not favouritism or the risk of favouritism. So, and we need to have some kind of things that are not measurable. We can’t do procurement without that, but it’s how we’re going to go about it that’s important. And right now, because we don’t have any court rules, court decisions on that, it’s quite free. And we have seen that, at least in Denmark, some criteria which are very, very flexible. And I don’t think that if the question is put before the Court of Justice, that they would say, well, just go about it. You can do whatever you want. I think that they would put some kind of guidelines into place where you have to ensure that there is no risk of favouritism.

Willem Janssen [00:29:59]

And I think just to add to that, I think also going back to what Marta said about the experts. I think in general; you can talk a long, long and hard about what an expert is. But in general, I think an expert has developed a certain methodology in their head to be able to assess good from bad. Right. And in a way, that’s what we’re seeing a lot in Dutch practise. Just talking about the coffee example is that, you know, employees of municipalities are sent in the evaluation board and clearly everyone always complains about coffee in their workplace. But what they try to do is to make that more objective by simply, say, stating, these are the criteria that you need to look at. So, leave your own like biases behind. Try to do that and actually say is this, you know, is this a sour taste or is it a bitter tone or is it actually sweeter or is the temperature nice? Right, or is it you know, can you drink lots of this? Is the coffee too strong in terms. And as I’m saying, these some are more verifiable than others.

Marta Andhov [00:31:01]

And nice is not particularly sort of verifiable.

Willem Janssen [00:31:03]

Exactly. Yeah. So that’s the hard bit. But then you can work in scales, right? You could say very nice, nice, average, etc., etc.. Nice is definitely not the right word. But like, do you understand like that? I think in a way it’s not just experts is one route, but evaluation committees to try to make things more objective can also be done by laymen.

Marta Andhov [00:31:24]

I think as long as you take it out from the ones that evaluate. So, I think that when you have those criteria, you need to sort of outsource it. So, there is no again, the risk of favouritism that Rasmus mentioned, then you can do that. But I just want to have and it’s nothing to do with law, but I just wonder because if you are a listener and you’re thinking, oh yeah, great, those Dutch and those Danish, they have plenty of money to do those things. I think that there is also something to address here that, you know, some municipalities or smaller contracting authorities of different member state, that the budgets are even smaller. I wonder whether some of those things that we discuss right now. It’s actually possible to be carry out, you know, to for example, you know, employ an expert, you know, three to evaluate taste of something. I think that that is a good approach. I’m just wondering if this is not something that some of our listeners might be. Yeah, not possible in my member state. And what then to do because then is that meaning that we cannot use those type of criteria and then we going back to the argument that those member states will say that’s why we just need to use the lowest price, which I think it’s something that none of us really would want to see in years to come.

Rasmus Horskjær Nielsen [00:32:36]

Yeah. I agree that that could be a valid point, but you could also say what if the contracting. I thought you doesn’t have the resources to look at the market. Because they didn’t look at the market, they can just do whatever?

Marta Andhov [00:32:51]

Yeah, that’s a fair point.

Rasmus Horskjær Nielsen [00:32:52]

I think that you need to fulfil the requirements and there will be, in my opinion, and that is what I stating in the PhD thesis, that there will be requirements in order to ensure that you don’t have the risk of favouritism. And even though there isn’t a court case right now, there are other court cases where they look at it and see that, well, as a main rule, you have some kind of flexibility, but you have to put something into place to ensure that there isn’t the risk of favouritism, and I’m just employing that structure in relation to award criteria.

Marta Andhov [00:33:33]

I just have one last question before we jump to the dessert and it’s a bit cheeky. So, Rasmus, when you think and when you discuss competition. Because in my eyes, let me just guide you through my line of thought. And I really wonder what you think. From my perspective, if we look at the competition as a counterweight to favouritism. The competition can be still narrower than the very broad almighty. We need open competition to the broadest concepts each and every time in public procurement. And I wonder whether which one of those you kind of prescribe to in your thesis?

Rasmus Horskjær Nielsen [00:34:17]

If I understand your question correctly, you need to have a competition that fulfils the requirements and that can as a result, you can have a quite narrow competition without it being a problem in relation to the sort of procurement directive or the principles. I think that if you have to open the competition now, other tools you need to use. So, for instance, we have them and that might be taking one of the questions up now is that it’s right now you you have procurement documents in every language in the EU. But it might be able to and that might be looking ahead. You might be able to have some kind of track translation of the procurement documents, and then you can put forth a bid in your native language, and then it’s going to be translated. So, the contract authority can see if that would be a tool to open the competition. I don’t think that small tweaks in the competition actually will…

Marta Andhov [00:35:29]

So you see rather the bigger issue of language just as an access point, for example.

Rasmus Horskjær Nielsen [00:35:33]

Existing is a great barrier and I think that’s the point we should look at and not the small tweaks in rules and regulations regarding competition.

Willem Janssen [00:35:44]

Okay. If you can ask one cheeky question, I can definitely also ask that and then we’ll promise to go to dessert. As I said, this one part of your thesis that I would didn’t want to leave unnoticed. And I think it comes to the transparency requirement for for scoring models and methods. And the reason why I was sparked by it is also because of the research that former and current colleagues of mine at Utrecht have done [00:36:05][…]. [0.0s] And they’re very clear. They basically say it is and I’ve had them use the word it is stupid.

Marta Andhov [00:36:16]

I heard that word too.

Willem Janssen [00:36:17]

Exactly. It is stupid to not publish. That’s by the way really only economists can do something like that. It is stupid to not publish the scoring method.

Marta Andhov [00:36:30]

Or not to require them by law.

Willem Janssen [00:36:33]

That’s the next step. So, basically now in the Netherlands, they’re working with the non-obligatory nature of the law when it comes to this. So, now they’re saying as a contracting authority, you have the room to publish it, so you should. And their line of thought is, and perhaps I’m briefly paraphrasing now, is if you don’t give bidders the model they don’t know how to make come up with the best bid. So they have no clue about how you’re actually going to apply the weighting and the criteria that you’ve published. Right. So they can’t go onto their Excel sheet and give you the best bid. In response to that many contracting authorities say, well, but then they know how we’re going to evaluate. Right. There’s no leeway anymore for us to actually judge them. Whereas I think and I would fully agree with them, I think the argument is your what you want to get the most out of the bids. So of course, you want to give them as much information as possible. You want them to be able to say, I’m actually going to up my price a tiny bit because then I’ll get or I’m able to offer more quality. And that gets me this much, these many points. So, it actually helps me. And then if I can just pose that towards the law, then EU law says you don’t have to write the master case that you mentioned already. And this is where I think your well, one of the more feisty comments or conclusions in your in your PhD comes in is that you actually advocate for transparency. And I want to just give you the floor to talk a bit about that.

Rasmus Horskjær Nielsen [00:38:03]

Yeah, that’s correct. I only looked at the the problem from a legal point of view, but whereas economics they look at what should you do? I look at what is the application, what do you have to do? And I agree that the [00:38:21][…] [0.0s] Case it state is quite clearly you don’t have to publish the evaluation method. The court is also saying that while you have the award criteria and you have the weighting, so as a bitter tender you have all the relevant information. But and it is also stated in the [00:38:41][…] [0.0s] Case that you have to have a stable weighting so you can look at the weighting and then you can rely on that. In my opinion, it’s a bit of a false premise because it is very difficult to control that, and evaluation method doesn’t affect the weighting. The choice of evaluation method is actually affecting the waiting and then the information for the bidders. In my view, it could be relevant to disclose the evaluation methods beforehand, as we also do in Denmark, because it’s not really possible afterwards to control that the weighting is upheld. And that is also why we go into like the control board competition, whether in the Court of Justice they state that you must set the evaluation method before you open the bids. So, in that way you can control that there is no risk of favouritism. But in saying that, they also acknowledge that you can use the evaluation method to affect the weighting and then affect the result of the competition. So actually, it’s getting a bit technical, I know, but in, in that way you have an acceptance that evaluation method does affect the weighting and if it does that, then you need to transfer and disclose it all. Then you have the relevant information as a bidder.

Marta Andhov [00:40:16]

And I think that this work of the fellow colleagues, the economist, the Willem you mentioned, I think that this is actually I’ve seen Jan in a couple of workshops really showcase mathematical examples how this thesis the the Rasmus underlines is flawed that that’s actually correct argument because he showed on several occasions that depending how you actually do the methods you can really affect. And I think he goes through like two or three examples and each time someone else wins, depending how you sort of tinker with the with evaluation methods. So, I think that this is another example of how the economics and law, again, must work together, because this is actually a very non legal argument that proves that I think legal thesis for sure.

Willem Janssen [00:41:05]

And I think anytime lawyers and economists tend to agree, we need to do so much. We need to start looking at Dessert. Dessert will be your advice practically. Right. So, you’ve finished your you’ve done the hard work. You’ve gone through all of the trials and tribulations. If you could very briefly give two bits of advice. If one’s also fine. But to future PhDs or current PhDs, what to do, what not to do? What were your lessons learnt? Can you share some of them?

Marta Andhov [00:41:38]

Or in other words, if you were to do it all over again, what you would do differently if there is anything that you would do different?

Rasmus Horskjær Nielsen [00:41:45]

Yes, there is a lot to say. A lot. Yeah. So, but the top two lists, I would say that that’s one thing I would do more often and that is talk to people in the academia about my thesis because it’s very, very helpful. And I often say that public procurement law is the nicest area of research because actually you often have a bit of, you feel that is bit of a burden because we when I talk to you, Marta, for example, I just require information and I’m talking about my thesis. So, I think that I’m hogging your time. But it is very effective just to talk about ideas and how you’re going to structure a thesis and so on. It’s very helpful. I should have done more of that.

Willem Janssen [00:42:44]

And so one was

open to others and discuss your research.

Rasmus Horskjær Nielsen [00:42:47] Yeah. Especially in public procurement because everyone is so nice.

Willem Janssen [00:42:53]

Except for…

Marta Andhov [00:42:56]

Working on the thin line.

Willem Janssen [00:42:58]

 Except for Willem Janssen.

Rasmus Horskjær Nielsen [00:43:00]

And the second part is that I should have had a better plan for the final half year of my thesis. I should have had a final draft of the complete thesis finished earlier than I did because I have a very understanding wife. But the last couple of months they were very gruelling on it. It’s a PhD. Yes. It’s quickly become becoming a family project and it’s the last couple of months there are. Yeah, they were quite difficult so and I think I if I had a like a final draft a month earlier, it would have been a bit more easy, the final couple of months.

Marta Andhov [00:43:44]

So when you think if you would just sort of shoot a line, so we talking about three years PhD just for because in different countries again a bit different. So, when, you think, what three months before to have a final draft, what do you think?

Rasmus Horskjær Nielsen [00:44:00]

Yes at least. Yeah, I would save four or five months.

Marta Andhov [00:44:05]

That gives you a good time to sort of revise and sort of clean it up and so on.

Rasmus Horskjær Nielsen [00:44:10] Y

eah. Because you don’t, you know, when you are you still are writing, you’re going down the rabbit hole and then you just eat up a lot of time and then you need a lot of time at the end just to read that again and again, try to structure your arguments and so on. So, and I, I didn’t have quite as much time as I probably needed it.

Marta Andhov [00:44:33]

Well, either way you did a great job. I was there. I listened to you and the book, and the project was great. And I’m very lucky that you managed also to publish it. And we look forward. We lost Rasmus a bit in the academic sense. But we hope you stay around, and you will pop by the procurement scientific field as much as possible. So, thanks so much for joining us today. It was great pleasure to have you on Bestek.

Rasmus Horskjær Nielsen [00:45:00]

Thank you.

Willem Janssen [00:45:01]

For sure. I promised I would say your last name again. Right. So, Rasmus Horskjær Nielsen, did I get better every time?

Rasmus Horskjær Nielsen [00:45:09]

Getting better every time

Willem Janssen [00:45:09]

You know, you’re just being too kind. All right. Thank you so much, also from me, this was Bestek the Public Procurment Podcast.

About Bestek [00:45:18]

This was Bestek, the public procurement podcast. Do you want to contribute to today’s discussion? Then share your thoughts on LinkedIn or Twitter. Do you have an idea for a future episode? Write to us at www.bestekpodcast.com.

Your content goes here. Edit or remove this text inline or in the module Content settings. You can also style every aspect of this content in the module Design settings and even apply custom CSS to this text in the module Advanced settings.