Willem Janssen [00:00:00]

Welcome to the Bestek, the public procurement podcast. Today, Marta and I are discussing the difference between framework agreements and public contracts and how to deal with criticism in academia.

About Besek [00:00:16]

Welcome to Bestek, the Public Procurement Podcast. In this podcast, Dr. Willem Janssen and Dr. Marta Andhov discuss public procurement law issues, the love of food and academic life. In each episode, Willem, Marta and their guests search for answers to intriguing public procurement questions. This is Bestek. Let’s dish public procurement law.

Willem Janssen [00:00:41]

Hola! Back at it again.

Marta Andhov [00:00:44]

Yes. Getting the work done.

Willem Janssen [00:00:46]

I’m very mindful that I shouldn’t laugh at the beginning. So, I didn’t laugh.

Marta Andhov [00:00:50]

Yes, very, very serious.

Willem Janssen [00:00:51]

Yes, very serious. So, let’s get right to it.

Marta Andhov [00:00:55]

What are we talking about today?

Willem Janssen [00:00:56]

It’s good to be back. We’re talking about, I think, a very topical issue today. The difference between a framework agreement and a public contract.

Marta Andhov [00:01:07] Heavy main course?

Willem Janssen [00:01:08]

Yeah. It’s very nerdy today. I talked to my wife about this podcast. She thinks we’re the biggest nerds; it’s nerd level nimbus 2000.

Marta Andhov [00:01:20]

It’s interesting that you say nerdy. I think that is quite practical. There is a lot of practical issues in differentiation between the two. So also interesting, different perspective on the topic.

Willem Janssen [00:01:31]

Yeah, yeah, yeah. I mean, if you keep telling yourself that it’s not nerdy.

Marta Andhov [00:01:36]

I was making this time a call to our dear listeners who are practitioners who will listen to this. Please support me. Please support me in comments. Let us know. Isn’t this a practical issue?

Willem Janssen [00:01:47]

Nerds can also be super practical folks. Anyways, Sorry, we promised we were going to get very serious today. So, we’re talking about this difference and it’s something that you’ve been working on recently and that we’ve been discussing for a bit. So, we thought it would be interesting to also talk about it in the podcast. For dessert, we’re talking about something that I would [00:02:08][…] [0.0s] Suggested to us when we called it out, What should we discuss for dessert? And a couple of people responded, and he said, Well, one of the things that I would love for you to discuss is how to deal with criticism following your work. Right, or based on your work as an academic, perhaps. So that’s what we’re doing for today. And so, we’re on the one hand looking at this more fundamental difference between the two sets of agreements or contracts. That’s, I think, one of the things. But we’re also looking at a more specific situation for which this difference is important. That’s why it’s relevant to combine the two is if you as a contracting authority can call off another framework agreement within the framework agreement. So basically, you have one framework agreement, you have one or multiple bidders that are entered into the framework agreement or are allowed to be part of it. And with that selection, you sign another framework agreement. And the third step would then be to have, you know, contracts that are given perhaps based on a mini competition or close. Right. But perhaps based on this general introduction, you can take us along a bit in in the importance and how you get acquainted with this topic.

Marta Andhov [00:03:33]

Sure. Well, why is it important? Why I thought and I tried to convince, as you can hear the listeners successfully, Willem, that there is a relevance and importance in discussing this topic. Well, it’s something that, of course, at times in practise it is an issue. And you can see in a variety of jurisdiction where there are national court judgements on the debate, what is the difference between framework agreement and public contracts. So, there is this seems to be a certain confusion. At the same time, a need for clarification. What’s the difference between falling from the practise? And I more specifically have been invited by a dear colleague [Name] who is a fantastic procurement legal procurement specialist in Sweden, to talk about her organisation about this specific aspect of awarding framework agreements. Within the framework agreements, as in Sweden, there has been a caseload that sort of flipped certain things upside down and that sort of follows and there is also Danish case, and I would try to sort of map it out a little bit today. But to start from a general standpoint on why the differentiation between framework agreement and public contract is relevant. Well, it is an application overrule question. So, if we have a public contract, if we assume that we award public contract, if you have some sort of follow up, a second stage following public contract that already stands outside of procurement rules. So, you have much more flexibility, more possibility of doing different things. It’s not as rigid as it is if you have as a first starting point a framework agreement. And just second step of that is a public contract, then up here you are still within those two stages. You are covered by the procurement rules. So, I think that this is the first step because there is a certain differentiation. The second aspect of that is that, of course, if as we as we all know why, there is no limitation for how many years you can conduct a public contract. There is a limitation for how long you can have framework agreement, and that is just four years. Right. So, again, this differentiation between those two will have a consequence because if you concluded something, naming it a public contract, but it’s actually a framework agreement and is for, let’s say, seven, eight years and there is not due to duly justified reasons you are in the violation of procurement rules. So, this is, again, a question of application of the of the rules. And as I mentioned, according to some case law, particularly, we have some cases that I want to share with you today from Sweden and Denmark that says that you cannot award framework agreement from framework agreement, but you can award framework contract or a ongoing purchasing agreement. And then my head sorta explodes.

Willem Janssen [00:06:45]

In my head. When you suggest to this topic my head start spinning as well. Perhaps I’ve got fellow Europeans on my side in this episode because my mind when bonkers when I heard about this topic. So, I think the starting point of that was, is I’m just posing the question. Don’t feel the need to answer it yet because I don’t want you to you know, you want to talk about the case a little bit. Like my first question would be, why on earth would you want to do this? This is the first one. And to this distinction that you talk about in in the Dutch Public Procurement Act, a public framework agreement is a public contract. All the rules simply apply. And the good legal reasoning for that is that the directive says things like, well, to avoid one, you need to follow one of the specified procedures in the directive, right? So that a there’s a link there as well. So, this this difference was already. Also, when I talk about because generally when I feel challenged and I think this can’t be, I have learnt to question my own ability and then to think I need to discuss this with people. And so, I spoke to some practitioners as well and they also hadn’t heard about this type of practise, or at least it doesn’t, it hasn’t come up yet as a, as a thing in the in the Dutch jurisdiction. So that’s why I was super interested to hear that there is some type of one practise going on in other member states and to that this case law rate stuff that we can discuss.

Marta Andhov [00:08:17]

Exactly. And this is again, I’m following the lead of Willem in that sense for any of the practitioners that listen to us and have some viewpoints or have some caseload that will be relevant here, if you could let us know, that will be fantastic. Then maybe I can gather all of it and prepare some sort of blog post for our website that will include more comparative vision of what’s going on in Europe in context of this. But yeah, let’s, let’s start because I think that then I already have so much question about the Dutch approach, but I’m going to restrain myself.

Willem Janssen [00:08:54]

 I know you want to restrain yourself, but maybe to explain that a tiny bit further. There is one link to something that’s come up in recent years is this two-phase type of contracting that happens in the Netherlands where you have two phases of contracting, one to sell the tender and second to two to actually start creating. So, it’s more of a design phase and it’s outside of the traditional ways of competitive dialogue. But one of the issues they’re faced with here, and I’m sure you come back to that, is how specific do you need to be at the start. How specific should it be?

Marta Andhov [00:09:29]

So, exactly in context of all this discussion that we’re having today, there are two things that we trying to really answer, because those are the ones that I think that pose a certain issue. One is how specific you need to be. And second, the second of them is do you need to have or at which stage or which point you need to have something that establishes an obligation to purchase at least minimum requirement because. Let me restrain myself because I feel like I have a quite nice idea for a structure, and I got excited willem said three words about Dutch thing and I already want to ask a bunch of questions.

Willem Janssen [00:10:08]

I know it’s like a nice thing to blame the Dutch, but we’re not all bad.

Marta Andhov [00:10:15]

Exactly.

Willem Janssen [00:10:16]

Keep going. Follow your original structure. I promise I won’t nudge you.

Marta Andhov [00:10:20]

We’re going to get to the Dutch, too. But let’s start with that. We’re going to start with the Swedish case law. As I mentioned to you, this is sort of where what got all the ball rolling. So, I’ve had a pleasure of of working on framework agreements for some years. I already published some things on, on the subject matter of what’s the difference between public contract and framework agreement. But I didn’t look into this framework agreement of framework agreement. And then I got to know about this Swedish case law, which is the judgement from the Administrative Court of Appeal in Stockholm. So, this is not the Supreme Court, but is the Appeal Court. But ultimately what the court established is that all awards from a framework agreement must result in a contrac or in contracts. And those contracts need to settle all conditions. At least for more than 50% of a contract value. So, it gets quite specific, right? So ultimately, on the basis of that contract or sorry, on the basis of that judgement, our awarding framework agreements from a framework agreement is not something that Swedish court practise allows. So, what that means in practise? In practise, that means that you cannot call off from a framework agreement with one or more suppliers because there is no differentiation here in a ranking order. And IT system, including, for example, implementation services. If the services cost more than the system itself and they are not specified to a full extent and by full extent you mean here exact volume to be purchased, right? According to the Swedish courts, this type of setup that I just described is a framework agreement. And that is not allowed. So this already sort of is interesting. But then there is another administrative court of [00:12:33][…]. [0.0s] Now I need to try to pronounce that [00:12:37][…]. [0.0s] I’m very sorry to all the Swedish listeners for that pronunciation, but what that judgement does is that they kind of play the mixed procurement approach to them. So, they’re saying, okay, you have, you have an agreement that ultimately have both elements of what you would traditionally classify as public contract. And how they differentiate here is that they saying you have certain part of this agreement that has a binding obligation. So, you bind yourself to purchase, let’s say, minimum 20% of the value of the contract with the option of buying more up to the market value of the framework agreement. Kind of straightforward here. That’s that’s a general approach that you did you follow. But what the court ultimately said in that case is that, well, you need to differentiate between them applying the rules on fixed contracts, identifying what they refer to as or this dominance principle or the principle of main purpose of contract. There’s a slight differentiation between those two, but to be very honest, I’m not entirely sure how it rules. So, we might need someone to clarify that for us. But would you then do is that out of the agreement, you’re saying, well, part of it is a contract, part of its framework, depending on which one has a higher value, which one is of the main purpose. That’s what the agreement in the end of a day is, which again sorta of makes the idea of having a framework agreement in the first place a bit bizarre. Because in particular sectors, you may need to have a minimum sort of required purchase rights to kind of keep a certain company on your framework. So those are the the Swedish practises. So, in general, you need to be super specific, a very traditionally understood the notion of you have a framework agreement, your word contract on the basis of that, it needs to be very specified. And this seems to be as something that particularly the largest buyers are interested in, to have a more flexibility to be able to, you know, specify things. Because ultimately, what we when we think about it, we are thinking about this notion. You set up a first framework agreement that is of a very general nature, and then you are setting up a second framework agreement within the original four year. So, this is an important thing to have in mind, right? We are not talking about situation. Did you have a framework agreement for four years and then three years in your establishing in the mini competition, not a public contract but another framework agreement for four years?

Willem Janssen [00:15:20]

Just the same would be, you know, if you would have a normal framework agreement and just two weeks before the end of it, you start giving out all these contracts. Well, I think courts would be hesitant, even though they were based on the mini competition, whatever, you know, they would far exceed the terms and conditions of the framework agreement. I think courts would even be hesitant to accept that. So, let alone awarding a framework agreement that then would extend as well with the other contracts that are given.

Marta Andhov [00:15:46]

Well, you see, I mean, to say that I wouldn’t agree with you on that. I think that even if it’s quite laid in a framework agreement, but you award a contract, I think we have a clarification of that in one of the residents of the directive that your contract can surpass the timeline of the framework agreement. This is not an issue there.

Willem Janssen [00:16:09]  

If they’ve run for far longer than that, there needs to be a certain amount of reasonableness around that.

Marta Andhov [00:16:17]

Yeah, for sure. What you cannot do. I think that there is no point of any questions or doubt. You cannot do another framework agreement. And that is specifically connected, of course, with the notion of open competition. Framework agreement limits the competition. So that’s the reason that it’s sort of locks your market. So, the only can do that for specific time. So, the Swedish practise is very conservative and as I mentioned, the largest buyers in Sweden are sort of arguing that there is a need for flexibility because there is a good business case to be made. Right. So of course, then the moment when something happens in Sweden or in Denmark, they look to each other, whether there is some sort of support on on the one way of arguing or another, because they share a lot of sorts of history, sort legal culture and so on, so forth. And then in Danish context, this is also when it becomes slightly, slightly complicated because on the one hand side, we got quite old case from 2011, the [00:17:21][…] case [0.5s] versus Ballerup municipality. And in that specific case, from 2011, the Complaints Board, the Danish Complaints Board said that it’s not possible to award a new framework agreement out of existing framework agreement. And what they referred to was what was the wording within the tender, right? So, within what was referred to as a public contract, the wording that was used was that the contracting authority was expected to purchase large number of goods in question. But without committing to some concrete number. And then there were some other aspects such as that the subcontractors in question would have a duration of two years, and then the extension was allowed twice for two years more. But also, this notion that the contract was to allow another municipality to have the possibility to join the agreement subsequently. So, all of this, the complaints board said this is a framework agreement, this is not public contract, and you cannot do framework out of framework agreement. So that seemed that at least on Danish market in 2011, this clarified the situation. No go to such a situation. But then with the new directives and an implementation of new Danish Procurement Act, in Denmark, they were working quite hard on preparing the so-called preparatory work. Do you have preparatory work in Netherlands, too?

Willem Janssen [00:19:01]

It depends, Well, we have we don’t have preparatory work. To a certain extent you would have the deliberations in parliament, which is often called preparatory work, but has no legal standing, legal basis, I would say, to as a as a legal source. But you would have the explanatory memorandum, which is kind of more which are more recitals.

Marta Andhov [00:19:22]

Yes. So, this is quite the big document that somehow is to lay the intentions behind the rules, so to speak. And then in many ways, a lot of things that you kind of didn’t manage to put into the law you put into the preparatory work. Right. So, in the preparatory work to the new Danish Public Procurement Act, there is actually a passage that predicts in principle allowance of calling framework agreement of framework agreements so goes against what earlier on the Danish complaints board indicated. And I will quote you right now this is please dear listener have in mind that this is not anyhow official translation of Danish to English.

Willem Janssen [00:20:07] I thought you were going to do it in Danish.

Marta Andhov [00:20:09] Yes. I don’t think that I would get away with that. But thanks to to to my dear colleague Rasmus, who helped me out and looking through this. The sort of loose translation, so to speak, would, say the following. Contracts awarded on the basis of framework agreement may have the character of continuous cooperation, also called ongoing purchasing contracts or framework contracts. This means that the contracting also can award a framework contract on the basis of framework agreement, which only states an estimate of expected purchase of the agreement, and which does not necessarily entail an obligation to purchase a certain number. So, this is really interesting, right? Because from from the conversation that I had on the subject matter, it seemed that there was again, similarly a bit as in Sweden, there’s really a substantial political push to have a solution like that. At the same time, apparently way while this preparatory works was draughted, there was a conversation, informal conversation with commission to check whether this potentially might be a problem. And the commission was quite welcoming to that solution. So, it seems that that was kind of okay. And that’s the reason that that ended up in the preparatory work. But there is no follow up case that will give us information whether the sort of the complaints board would change their opinion and the preparatory work. Usually when it comes to really Danish, Danish issues of law is very important and is often referred to. But when it comes to a provision that are really based on the EU directive, then they usually go much more to interpretation of the directive and go much broader than the preparatory work. So, the preparatory work to some extent might be not upheld. But what on the basis of this of the Swedish and Danish cases, what really comes to to mind in the language that is used where I see the main issue and then where the Netherlands practise also comes into place. The language that is being used is public contracts. Framework Agreement framework contract. And it’s not very clear what is the distinction between them and also the distinction that follows from, for example, something like the Commission’s interpretive communication on framework agreements from some years back. That interpretation seems to not be necessarily followed by the national interpretation. So bear with me for a second, I’ll just give you the general differentiation, and then we can dive into also the questions of Dutch practise. And I’m very keen to hear what Willem thinks about it. So if we look at the this commission’s interpretive communication on that basis and on the basis of at least my logic, that is supported by literature review, of course in the research that I did within this area for some time a couple of years ago, Framework Agreement, as a general standpoint, it always will depend. We need to analyse them individually how they are written. But in general, as a concept framework agreement does not create an obligation on the other of the party, so contracting authority or the supplier to actually perform the subject matter of the agreement. So it does not create an obligation for me to buy and for the supplier to supply its sets as general terms of our engagement in the years to come. Right. This is quite opposite, opposite to public contract, which creates an obligation there is an obligation for me to do something and for to pay for the goods, let’s say that I receive. And then the question that really stands up there is, what the heck is framework contract? And if you look is that public contract or is this a framework agreement or it’s something else. And this  interpretative communication from commission ultimately just sets out that framework contract is a framework agreement that sets out all conditions for future contracts and is a type of legal instrument for the automatisation of the award in the second stage of framework agreement. So ultimately a good example of this is if you think about multi supplier framework agreements with direct award or so-called off shelf purchasing where you do not reopen a competition to many competition that is framework contract.

Willem Janssen [00:25:05]  

I find it’s more confusing but yeah. Please, please go ahead.

Marta Andhov [00:25:11]

This is exactly that. This is pretty much it. This is where I’m sort of going back to this to this Danish preparatory work that says ongoing purchasing contract or framework contract because they use term framework contract, which on the basis of which you have an estimate of expected purchase and which does not necessarily entail an obligation. So, they said you don’t need to have all of the circumstances issues. Right. And then that brings me back to the Dutch practise, Willem, because you said framework agreements are public contracts, right? So, is that meaning that you will always, when you establish a framework agreement, have an obligation to purchase within the framework?

Willem Janssen [00:25:56]

See the thing is it’s really interesting but this difference and I don’t think so. But the difference is just not seen in practise. Okay.  I follow your reasoning. Right. But in a way I wonder why is it. So, we making life far more complicated by having all these different types of definitions. That would be my first step. And the question is. What’s the problem if we make it a public contract? Maybe I’m simplifying too much, right? And maybe I’m not overseeing the consequences. So I’m trying to, like. I’m just genuinely asking the question because. They say, let’s just assume, right, in a hypothetical world where, you know, Dutch law is the law. You would be under a duty to tender a framework agreement because it’s a public contract. And if you’re in a framework agreement after that, you have called off so many competitions that let’s call them contracts, right? They’re not a framework agreement. They’re just a contract that you sign based on that framework agreement. I find that in my head makes a hell of a lot more sense than calling the framework contract. Because it’s just a contract. Right. And maybe I’ll let you because the listeners can’t see what I’m seeing, but I can see steam coming out of your ears right now.

Marta Andhov [00:27:33]

That’s not true.

Willem Janssen [00:27:35]

Maybe you can shoot my bubble and just say, well, life’s more complicated than that.

Marta Andhov [00:27:41]

Well, you know, it’s not necessarily that I want to shoot your bubble, but I think that this is what I would be really interesting to hear, if that is from you or our listeners, is whether I missed something or I misunderstood something. But for me, okay, let’s start with this framework contracts versus framework agreements. From contractual perspective. There is no difference between framework agreement and framework contract. They do not create an obligation. So, if you call it potato, but there’s no obligation in contractual sense. For me, the main difference between public contract and framework anything is that you have an obligation. And I think that the reason why we in practise don’t want to or we explore, explore the possibilities of getting that obligation as late as possible is because we have an ongoing need and those needs can be fluctuating. Right. So, you’re trying to say, okay, at the beginning, we know that we need, you know, some sort of IT system, but we all need, in the example services and maintenance for them for sure. But we may need some sort of hardware to and supplies, right? Then you will within the four years one two may be established as other framework agreement that has more specificity and that sort of again gives you time to sort of get the supplier more prepared and get more understanding of what is on the market. And then you award the contract, right? So, I think that there’s a need for flexibility in this sense. Where my mind gets slightly confused is the following. Can you have a framework agreement or a framework contract? Let’s just bundle them together. That establishes a minimum obligation to purchase, but it’s not a public contract. I don’t think so. And then can you have a public contract, in which you do not have a minimum obligation to buy? Again, I don’t think so, because I think ultimately the two different one is you got binding obligation and there is specific value, right, that comes in. So what I struggle with, particularly also this passage from the Danish preparatory work on ongoing purchase contracts and this notion that you can enter into longer framework or this longer contract without mandatory requirements to purchase. So what, you would have a public contract and you say the contract is for, you know, 3 million, but I’m not obliged to buy and you’re not obliged to perform. And this is, you know, similarly, as you sort of ask me, this is a genuine question from us. Am I not seeing something? Is there a different ways in which how you structure a contract? Because for me, this is straightforward the difference between. And if you would want to simplify really the conversations that we’re having is that you trying to push the obligation to as far as you can. But can you have it? What did you think if you at the Dutch practise?

Willem Janssen [00:30:56]

Well, the more we think we talk about it, I don’t think. I think we’re not actually that far apart because clearly as a characteristic and sorry, I should have been I think clearer about that as a characteristic also in Dutch practise. The framework agreement offers that flexibility and allows you to call off whatever whenever a need arises that you have perhaps somewhat foreseen. But not entirely. I think the more of the reasons is why the Dutch legislature called it public contracts. Still is because you still need to tender it. Yeah, but I think the difference is still there. Right. But that’s still where I’m my mind is like continuously trying to process this distinction.

Marta Andhov [00:31:41]

Because it’s extremely technical. Because what you actually if you look and I did that exercise, if you look at the wording within the provisions of framework agreement and so on, so forth, we have defined what is framework agreement. We have defined what is public contract. But the problem of that is that sometimes the reference in the provisions on framework agreements just to contract or to a bid. Yeah. And then in other places, in some places there’s differentiation what they refer to as a framework agreement on a contract. And then you have other provisions under the directive where you use framework agreements and public contracts interchangeably. Example here is provision. I think it’s  72 on modification. And they refer to framework agreement in public contract. And they said, I’ll use here, you know, kind of for the sake of the same meaning, kind of the same not meaning the same, but the rules applying to them the same way.

Willem Janssen [00:32:38]

Yep.

Marta Andhov [00:32:40]

And just to make it more complicated, even that we really don’t have to, and I will promise to wrap it up. But I was also really interested when you mentioned about this sort of almost like procedural contract, this two-stage contract, because the years ago when I started to discuss also the question of framework agreements in Poland, right now, the framework agreements picked up slightly in the Polish jurisdiction. But some years ago, when I started looking into that, they were not used at all. And but surprisingly, people would describe to me this conditional contracts. And the conditional contracts will be kind of exactly what you describe. That is like a public contract with two phases. So, they saying that it would be normally awarded contract, but it would be very defined until specific certain circumstances. Would they or cure it would establish an obligation to purchase. So, you know, this is slightly different because also when we’re talking about mandatory character or obligatory character or framework agreement, there are two elements here that we need to distinguish. One is whether you are allowed as a public, as a contracting authority to go and do your own tender, or you have to use the existing framework agreement that somehow you are obliged to use. That’s a one sort of aspect of this mandatory framework agreement. Another one is if you have a type of agreement that sets certain minimum rights, it says this framework is agreement is for five mill, but you need to buy at least for one mill. But the max value is five, right? Let’s say. And then isn’t that a public contract? What do you think? I kind of feel like this fulfils the specificity of a public contract, that you can have something that is a framework agreement, but it’s at the same time public contract. Are you saying that I’m going bonkers?

Willem Janssen [00:34:37]

I’ve never dared to say that. And I think just because your question was also just to reflect on the Dutch practise so this would never be with multiple contracts is its very construction related. So, let’s design a solution and then the second phase will actually contract whatever we’ve designed together. Yeah, there’s also this development on using construction teams. So, it’s more about how to make sure that the price that we pay for the products is actually because there’s so uncertainty of what we’re getting. So, that’s more the issue. So, there’s no question that if it’s a framework contract or not, it’s a public contract. It’s just it has the same. Well, not the same, but has similar uncertainty related to to whatever happens after the signing of the first contract. Right. You say with the framework agreement, there needs to be a certain amount of certainty. And one of the things that comes up also in this in this setting is the modification of contract discussion. Right. Is that a is that a problem? But also in this step, I would say perhaps it would be my last question to fire back at you or to fire sounds or negative statement just to give you the floor again. My first thought when I heard this, like just bringing it back to the framework under a framework contract is I understand this political push. It’s everywhere to create more flexibility within the legislation to just do whatever we want to. Because we’re doing it out of the kindness of our heart, and it works. Right. And in a way, I understand that because often it is the case. But as lawyers, we have to still protect whatever realm there is, you know, to also achieve the objectives of the law. So, my question would then be, isn’t this circumvention of the law if you really narrowed competition and you briefly touched upon it? But for Dutch practise, this would be a big argument to say it’s not possible if you really narrow competition in the first framework agreement and then, you know, you get to do whatever you want in the second one because you’ve managed based on vague terms and I’m being a bit more dramatic than perhaps necessary, but it’s very vague. You know, you just get the people involved that you kind of know already and then you move on to the next stage. So, could you shed some more light on that issue before we move to dessert?

Marta Andhov [00:36:49]

Yeah. And I think just as a three words to that, answer your question we definitely need at least another two or three, I think episodes on a framework because this is a lot of things is something that is quite used in practise, but it’s many unanswered questions still.

Willem Janssen [00:37:06]

And I also want three on in-house.

Marta Andhov [00:37:09]

We already did it.

Willem Janssen [00:37:13]

We already did frameworks as well.

Marta Andhov [00:37:17]

Yeah, to the point. So, yes, I think that this is a because the problem is that we need to be particularly delicate when it comes to framework agreements, right? Because it limits competition. So, I think that for sure, if it’s to be expanded above four years, this is a no go. But the question is what within four, four years? And exactly the point of the point that you are sharing, whether it’s very general, like it’s too general, it doesn’t really clearly specify what what that is. And from that reason that brings that comes another question to juxtapose to whether see whether argumentation is different. What if we flip it and we say dynamic purchasing system, out of which you call off framework agreement because up here you don’t have narrowing competition. Up here, you kind of put everyone on, right? I can see that your head is sort of spinning.

Willem Janssen [00:38:17]  

I wasn’t going to say this, but it also reminds me of like an open house system, like the taken in type of thing, where you you just let everyone in that fulfils certain criteria, you know, the doctor fault type of jurisprudence where I thought. I would say let’s leave these questions out in the open. That’s not complicated further with.

Marta Andhov [00:38:40]

So, we will, we will come back at some point the dynamic purchasing system and framework agreements. One last sentence to the point also that Willem made, and I think I would want at some point to come back is this approach of modification of framework agreements versus modification of public contracts. Just to say, I think we are still in competition, particularly if we have a mini, mini, mini competition to go for. So, I think that actually, in my personal opinion, the modification maybe need to be even more strictly interpreted that in public contract because you got still chance to choose someone else. So, that’s just as a sideline.

Willem Janssen [00:39:19]

You can’t throw bombs at the end and not expect me to respond. But yeah, keep going.

Marta Andhov [00:39:23]

I think what I’m going to prepare, because I got quite extensive notes from this, I’m going to do a blog post and I’m going to just leave it with a cliff-hanger and hopefully our listeners will find it interesting to come and read through the blog because there’s one more case in Denmark called [00:39:41][…] [0.0s] is from 2020, which again, so to differentiate slightly or point out the challenge was that something that was procured as a public contract, the bidder said, no this is actually a framework agreement. And the Complaints Board again, got in a little bit into conversation about, well, what is the difference? So I’m going to wrap it up, I’m going to write it down. And please come and join to our fairly new, freshly established blog that we’re going to have on our website and read some more. And with that, the main takeaway, it’s a mess. It’s a mess. It’s an absolute mess.

Willem Janssen [00:40:20]

You’ve been very insightful, and I’ve been challenged challenging you a little bit by posing some some very strong Dutch positions, I suppose. We’ve talked a bit about the difference between a framework agreement and a public contract and a framework contract and all these different types of agreements and contracts that complicate it further. Thank you for also highlighting super interesting case law firm from Sweden or Denmark that I, for one, would have never known about if it wasn’t for you. So, thanks for that. Back to the dessert in the last couple of minutes that we have. The main question is how to deal with criticism in in academia. And I understand that this is quite broad still, but let’s do it based on research perhaps. I mean, you could include teaching and just general activities. But I think the suggestion we got given was mostly related to your work as an academic.

Marta Andhov [00:41:15]

It was specifically, I think, related, if I remember correctly, from from comment from from Albert on our LinkedIn was specifically how not to take your research too personally and then afterwards how do you react to criticism. But it actually relates also slightly to, not necessarily teaching, but again mentoring and dealing with young academics because I think that this is also very important. On the one hand side, I think it’s it’s it’s actually not only for the receiver question for the receiver. So, how I am to react to the criticism that I’m being given, but I think is equally also other way around. So, like if you want to provide a constructive feedback that may include criticism because that ultimately what constructive feedback will be. How you do in a right manner, right. So, I think it’s sort of connected with the two. We sort of had a little bit of prelude, a bit of a chuckle before we started to recording on this that we attempting to answer this because I think it’s a very difficult question now how not to take too personally criticism to your own research.

Willem Janssen [00:42:26]

The thing is I think it’s a super valid question where the answers are also incredibly difficult because I get pissed off if I’m completely honest. And of course I perhaps wouldn’t show it at first

Marta Andhov [00:42:41]

Well, you are you professional you’re bubbling up inside.

Willem Janssen [00:42:46]

We talked about steam coming out of your ears before. I was joking before. I think twhen you’ve worked on something for so long and it hasn’t seen daylight yet, it becomes like a, you know, the metaphor of a baby comes up very quickly or the metaphor of like something that is just your brainchild. Like it’s really close to something that you’ve worked on and that you’re perhaps also proud of. And then to receive a criticism which is maybe in my head, the difference between feedback always seems a bit more positive than criticism, but I fully relate at least to the feeling that it’s very difficult. And I think, yeah, us as academics, I think sometimes we differ a bit from other professions where your integrity and standing as a person relies so much on what you put down on paper and how much your self-worth can also be be related to it.

Marta Andhov [00:43:47]

You are, in a sense, your own brand, so the research that you’re putting out is your own brand. And it means that much to a lot relies on that. So, it is it is personal and it’s difficult to deal with the criticism. But I think it’s again, also going back to what you indicated already, we also need to differentiate with different from different types of criticism, sometimes once in a while, if you for long enough in this business, you would just get someone who will stand up on a conference or something.

Willem Janssen [00:44:20]

We all know that person.

Marta Andhov [00:44:22]

We all we all know at least a couple of those occurrences in our life that someone just stands up and just is a bit mean or nasty and just wants to, I don’t know, get a bit of light on themselves or kind of gets off on the fact of kind of feeling superior or whatever. And I think that sometimes when that happens, it’s fully understandable that one would feel really negative and criticise and your self esteem goes a lot down, but I wonder whether we actually can. I think that you need to differentiate that from another form of criticism, which is that someone stands up and give you a constructive feedback. What I mean by that, there are certain standpoints that within our field we know more or less the sort of core group of people that in our community we work with. We know sort of where our normative standpoints stand in a variety of places. So, actually is very, very keen. And I hope that Albert won’t mind that I use him as an example because he also given that topic. But Albert and I, we normatively stand on, on quite different starting point in our approach to research and public procurement. So, it will be quite often that we disagree but the way. How we disagree the way how we like. One of the things that and it happened on several occasions before I was presenting something or submitting something I sent and I kindly ask Albert for feedback because I knew that he will be a, so to speak, the harshest critic because he seems things from a quite different perspective. And he always, to give Albert full credit that it very professionally, very kindly and you don’t feel like you’ve been wronged and the arguments that you can then use because it’s not necessary that you go in sudden to agree and change your viewpoints, but you know how to make your arguments stronger. And I think that this is something that we are to seek, right, because the constructive feedback can make our research better. But that’s the reason that I also mentioned that it’s similarly important to think about how you criticise somehow and how you give someone constructive feedback. So, it doesn’t feel personal. It doesn’t feel like you did a crappy job. It’s not that’s not what I’m saying. I’m saying this is very interesting. I think that you could improve it if you consider X, Y and Z or, you know, this one. Maybe I’m coming from a very different perspective, but I’m not fully convinced X, Y and Z. Right. We can agree to disagree in the end of the day, but it’s the form I think matters. Wouldn’t you say so?

Willem Janssen [00:47:10]

100%. I think it all relates to intention and form. So, how you you end up giving the feedback. On the other hand, if I can take a step forward. from from going from that, there can be scenarios when someone has done just that.

Marta Andhov [00:47:37]

And it still hurts.

Willem Janssen [00:47:38]

And it still hurts. I don’t want to be to overly dramatic about it, but I do think that one acknowledging so the form and like you mentioned it, I have also been at those trainings you do the hamburger technique, right, which I don’t understand because the best bit is, you know, well, I suppose that’s the thing. You start off with the bun and that’s a nice thing. And then the critique is the meat.

Marta Andhov [00:48:04]

We call it sandwich method. You sandwich the negative and positive.

Willem Janssen [00:48:11]  

So, of course that’s, that’s super important. I think that’s important mostly because it’s also time, right. If you, if you’re giving feedback on paper, it’s far harder and more time consuming this approach than to just give the feedback. Right. And just and then I know people sometimes say, hey, sometimes I can come across a bit harsh, but just know that it was because of time constraints and in a way that helps.

Marta Andhov [00:48:38]

I think it does.

Willem Janssen [00:48:41]

At least that’s one step forward. But then when you open the document and you, I get, you know, critique.

Marta Andhov [00:48:47]

Esclamation marks, that’s the worst.

Willem Janssen [00:48:50]

What the hell is this, you know, that’s not the stuff. This is wrong. You know, I think everyone’s been in that scenario, whether you’re a PhD or later on in your career, I think. How do you then the question is, is even though someone’s on it right, or they’ve not done it right, is how do you still step away from it? How do you, you know, make something positive out of it? Like what have you got ideas? Or how do you deal with it other than, you know, after the phase of smashing in a wall?

Marta Andhov [00:49:18]

Yeah, well, I think that, again, this is probably something that we also need to highlight that it’s very different when you’re dealing with because I sort of went straight away to the criticism or feedback depending, you know, how we worded in person or whether you touched upon something that I think is even worse in many ways, which is the written feedback, right? Let’s say you submit article or something and you get it back and oh boy, this is the worst, because I think that it’s very difficult to communicate kindness, as a purpose behind, you know, the sort of the criticism that comes on a paper. I think it’s a bit easier to do it in person. But the way that I will do it, I usually open it. My blood pressure will go to the roof and then I will leave it. I will leave it. And then I will try to go. And I’ll be very honest or for a wine glass in the evening or go for a run or something. I just need to leave it be. I just, you know, I didn’t develop yet a better scenario, but I need to go through the first immediate shock of, you know, my article not being accepted, let’s say, or something being said that this is totally off to the wrong streer, so to speak. And then but then, you know, one of the things that I pride myself in, I don’t know whether that speaks to, to, to our listeners, but I ultimately will comment that are, so to speak, grind back to the work, right? So, I was like, okay, yeah, this pissed me off and made me sad all that. But at some point, there is, you know, the saying that it’s not how like you start but it’s how you finish. Or you can let it break you and say, okay, I don’t look into that. I put it in their drawer. I’m I’m leaving that be. And that is in a way maybe a defeat sometimes it maybe can be also a success because you may just realise, okay, I need to rather cut my losses. I just sort of tried something. It didn’t work out. But if you strongly feel that there is something else, you need to go back to the drawing board, right? And that I think what ultimately makes you a great researcher is that you come back, you sort of face your fears, you read through it. And also, it’s perfectly fine to take what you agree with, but not that what you agree with, but where you see a value and you see, okay, yeah, that if I would do this and that, that will make it better. And things that you just think like, nope, nope.

Willem Janssen [00:51:46]

No, exactly. Because there’s also a reason behind, you know, finding a different venue perhaps for your paper then if it doesn’t fit this or perhaps it was just unlucky that you got a grumpy review or two.

Marta Andhov [00:51:57]

That also happens.

Willem Janssen [00:51:58]

The point that you mention about stepping away, that’s that’s valid. I still find, though, that the nibbles that you write, it still itches in the back of your head and you’re trying to be calm about it. But still, all you want to do is do something else. But I also find that letting the feedback, you know, it’s my brainchild. So, what I find is very useful is to give it to a colleague. If you have good colleagues, they read, they can read your article and then that they’re not emotionally involved at all. You probably wouldn’t be comfortable doing it with everyone, but like everyone has or hopefully has that type of connexion with someone where you can just say, hey, can you read this? This is the feedback that I got. What do you think? I’m pissed off, but does this help?

Marta Andhov [00:52:41]

Yeah, I did that a couple of times too.

Willem Janssen [00:52:43]

Yeah. And at the beginning of the, you know, particularly when I was doing my PhD, I kept a list of things that I thought I was good at writing, things that I felt like I had achieved. Because a PhD can be such a long thing. It’s a long period of time that you’re working on something with no milestones, right? Maybe there are milestones. And for me, having that list that I could go back to and kind of like reboot my confidence to a certain extent, I think that was useful. But putting it away and discussing it with other people that don’t have that emotional connexion is by far the most helpful thing that helps me to be honest.

Marta Andhov [00:53:25]

And I think, you know, to conclude, because it’s getting a bit long, our episode this time around is it usually does when we’re having a good time here. I would just say that also it’s again, I’m true believer and, you know, sense of community. So also, if that message can resonate with anyone, there’s no academic who does research that has. I strongly believe that, at least not any that I know that has not been rejected before, has not heard that, you know, whatever that person is doing is rubbish or it’s off or this like we all this is for someone. You know, my mum always said that to music for someone who, you know, really struggles with self esteem, this is not a job for we caught it because you just get pounded on your head time and time again and anyway, right. So, it’s a part that you need to develop a bit of thick skin and and also rely on the community around you of the people like Willem said, that you trust that sometimes if you get something like this. Let’s say rejection from a journal or something, that you can send it to someone else. And that happened to me. And then the other person will filter, and they’ll say, You know what? I read the feedback and I read your paper. I deleted half of the things that I really don’t think are relevant. For you. So now you have the thing that you should focus on, and you know that that helps too. I think.

Willem Janssen [00:54:53]

Yeah, for sure. And I also realised that like, these are only our experiences and I think they were that those are the things that we do to cope with this as relevant as I think they are probably to a broader audience as well.

Marta Andhov [00:55:07]

I think it’s a shared experience. It’s a shared experience.

Willem Janssen [00:55:12]

For sure. I mean, if I look at the number of rejections of grant proposals, I got, a couple that were awarded. Most of them were rejected. Some journal articles were rejected. Some were without any reviews accepted at another journal. It’s just part and parcel of the game.

Marta Andhov [00:55:28]

Also, not an objective process. I think that we also need to somehow for anyone who still believes that, get rid of that sense, it’s this is not objective process. The reviewer or whatever you do or someone that is providing you with criticism, that’s their opinion. It’s not given facts right. And that’s important to have. But at the same time, because I just feel that it’s important for me to add that last sentence on the fact that particularly if you’re younger, if you just starting and you may be writing some of your first stuff, listen to the more experienced colleagues, particularly the ones that you trust. Because what I starting to also experience lately and I need to say that I don’t think that that’s such a good practise either is that I started to see a certain sort of faction of people just disagreeing and disregarding and type of feedback, and I don’t think that that’s also good. This very tunnel vision is like, Oh, you’re old school, so I’m not. Listen to your viewpoints., this is my thing, which I think that there is also a need for a certain level of humbleness and context that if there are people that work on something 15, 20 years, they probably know a bit more and they for sure may have something valuable to contribute and feedback. So, I think that I would just want to leave particularly our PhD students who might listen to us also with with that message for me. So, I think that this was a pleasure. I really enjoyed our chat today.

Willem Janssen [00:57:03]

When you say these things and then we’ll definitely close up because we’ve been terrible at timing the last this one in the last episode, we promise to be better next time. But yeah, sometimes when you say these things, I think what does she think about the episodes where she doesn’t say that it’s a pleasure, does she just rock up at dinner at night?

Marta Andhov [00:57:33]

He made me talk about in-house again.

Willem Janssen [00:57:36]

Let’s talk sustainability. We might find each other again a bit more. Sure. Thank you for your time. It was a pleasure. And hopefully you enjoyed the conversation as well. If you can give us a like on whatever platform you’re using or if you can share the episode, that’s always much appreciated. And until next time, this was Bestek, the public procurement podcast.

About Bestek [00:57:58]

This was Bestek, the public procurement podcast. Do you want to contribute to today’s discussion? Then share your thoughts on LinkedIn or Twitter. Do you have an idea for a future episode? Write to us at www.bestekpodcast.com.

Your content goes here. Edit or remove this text inline or in the module Content settings. You can also style every aspect of this content in the module Design settings and even apply custom CSS to this text in the module Advanced settings.