This is the transcript of podcast recorded on 6 Feb 2024 with Dragoş Tudorache, a Member of the European Parliament and Vice President of the Renew Europe Group. Mr. Tudorache chaired the Special Committee on Artificial Intelligence in the Digital Age, and was a key force in the writing and the recent passage of the European Union’s revolutionary new AI Act.
Tom
You're listening to Pushing the Envelope: Life at the Cutting Edge of Customers Innovation. I'm Tom Mueller with the Cross Border Research Association. On this show, we explore the frontiers of customs creativity, in conversations with customs and logistics experts, technology innovators, research scientists and other leaders in the field. Industry insiders call the show “the Pencast,” because it's part of PEN-CP, and network for boosting customs innovation, funded by the European Union's Horizon 2020 Program.
Today I'm speaking with Dragoş Tudorache, a member of the European Parliament and Vice President of the Renew Europe Group. Mr. Tudorache was chair of the Special Committee on Artificial Intelligence in the Digital Age until the AI Act was passed last December. He also sits on the European Parliament's Committee on Foreign Affairs, the Committee on Civil Liberties, Justice and Home Affairs, the Subcommittee on Security and Defense, and the Delegation for Relations with the United States.
Hello, Mr. Dragoş Tudorache -- or Dragoş, if I may?
Dragoş
You’re very welcome, happy to be here.
Tom
Congratulations, by the way, on getting the AI Act passed. That was a long, complex negotiation that concluded with a marathon…
Dragoş
Not yet fully passed. It passed the Council last week, with unanimity I might add, which is good news. And we still have to pass it through Parliament for the final vote, but I don't expect any issues there.
Tom
Oh, very good. Well, half congratulations, and fingers crossed!
Dragoş
Thank you!
Tom
When is the final vote?
Dragoş
Well, we're voting in committees, because that's the sequence that we have in Parliament, on the 13th of February. So basically, very soon. And then the final vote, depending on when translations in all languages will be available, which is a bit of an issue because it's a very complex, and as you probably know, a big text. So hopefully we'll be ready with that for the March session. If not, at the latest in April.
Tom
Wow. Okay. Very good. I'll be looking for news on that. That's very exciting.
To start off with, could you give our listeners a word about your background and what brought you to the European Parliament?
Dragoş
Oh right, well, how far back do I go? I'm a judge by profession. So I started my career as a very young judge in Romania. I spent a few years doing that before I got interested in an international career, then I I worked for the OSCE [Organisation for Security and Co-operation in Europe], for the UN, in the Western Balkans, for the international justice programs there.
And then I moved on to the EU, started first with the European Commission delegation in Romania before the accession. And then after accession, moved to Brussels, worked for more than 12 years in the European Commission, where I led several big projects, mostly in the area of Home Affairs, but not only. [I] established an agency, did many other things. And then I was co-opted in government back home in 2015. I served as Head of Chancery and Ministry of Interior, then in the European Parliament. And here I am today.
Tom
Your background expertise and current interests in the European Parliament include artificial intelligence and new technologies, security and defense, internal affairs, transatlantic issues and the Republic of Moldova. I mean in various ways, each of these intersects with the topic of innovation in customs and border security, which is the focus both of this podcast and of PEN-CP, the EU-funded project to which this podcast belongs. So I'd love to explore each of these areas in turn and briefly hear your thoughts.
Starting with artificial intelligence and new technologies. You spent, as we’ve said, the last several years working intensively on AI legislation, and then the process of become one of the world's authorities on how artificial intelligence intersects with EU law. What in your view are the most important new tools in AI and machine learning that need to be implemented by customs administrations at the borders? And on the flip side, on the dark side, if you will, of customs and borders, what new tools do you expect criminals to be using to try to slip their illicit cargo past the borders undetected?
Dragoş
Well, like with everything, AI the power to optimize almost all processes that we employ right now. It's true, again, for almost every sector of human activity, certainly for every sector of administration. And therefore, customs and generally border processes, whether it's border security or processing of merchandise or persons, can certainly benefit from the employment of AI in any of these processes. It can simplify, it can speed up, it can do the sampling on the basis of algorithms that can be created on the basis of risk assessment. Customs is mostly done on the basis of risk assessment checks. So all that was already using, at least in the last decade, sophisticated software for any of that, and technology combined with this software. But now I expect that with artificial intelligence that is going to grow exponentially, both in terms of speed and complexity of the analysis and the risk assessments that are being done. So certainly there is room for use of AI. Again, both when it comes to custom checks, as well as border procedures in general.
You also mentioned the flip side, which is always present, again everywhere and certainly is going to be present here, because criminals are unfortunately, almost always ahead of the curve, sometimes ahead of the law enforcement authorities when it comes to the adoption of new technologies. They are usually early adopters than almost everybody else. So we can certainly anticipate that also they will be trying to employ AI to either disguise merchandise, or to counterfeit certain processes and procedures and paperwork and flows of documents. So that requires a response on the side of the authorities that will have to counteract, identify, detect, and respond to what would be, from my point of view, an easy to anticipate surge of the use of AI on the side of criminal organizations as well.
There is also another flip side. You didn't ask it, but I need to put it on the table, which is the flip side of potential negative effects of deployment of artificial intelligence in the border or customs context. Because in a way, we humans do have an automation bias. We do believe that well, if we employ a machine or algorithm to do something, in theory, it should be better, cleaner, faster. But it's not necessarily always that way, because you always have the possibility of discrimination creeping in, biases creeping in, whether they are existing in the databases that we use to train an algorithm, or whether they are engineered, in the sense, even without knowing, those writing codes might have blind spots in terms of effects of the algorithm towards, let's say, the ultimate customer of a process, whether they are companies that are putting their merchandise in the customs process, or whether they are individuals who are part of border procedures. So again, there is a duty of care when it comes to the effects of AI, to make sure that, again, they do not produce negative effects on the subjects of the law.
Tom
That's a very, very important flipside to mention. We do tend to think sometimes tend to think of AI at the borders, especially regarding human movements as a sort of dystopian, Big Brother... But you're saying that there are sort of implicit biases that can creep in, that we tend to trust machines more than we trust, in a sense, our own conscience.
Dragoş
Absolutely. Because ultimately, we will be asking algorithms to do what? To do profiling, again profiling of people, or profiling of containers or profiling of shipments, that might be riskier than others, and therefore, they need to be subject to controls or to certain fraud detection mechanisms. And again, when you do that sort of profiling, that sort of anticipation of risk, there is always that flip side, there is always the associated risk of creating discrimination, or checking or sampling some versus others, on the basis of elements and variables that may contain biases, prejudices. Either in the way, again, society has worked so far and that is reflected in the data sets that are being used to train the algorithm, or in the way it’s been designed by its creators.
Dragoş
It is interesting that we tend to think of algorithms as these things that are out there, but in fact, they encode some of our own biases. Human beings are writing those codes, and the way they think.
Dragoş
Always. And -- sorry to add complexities to it -- but we are also going to see, in fact, we are already seeing that is part of our reality right now, but it's going to grow even more in the coming years, which is AI algorithms produced by AI algorithms. So we are no longer in a world where only humans produce algorithms. That world is already by now yesterday, because now many of the very powerful foundation models, many of the large language models, are also capable of writing code themselves. So in the next couple of years, we will have AI producing AI. And therefore I think the risks for those blind spots increases even further.
Tom
This is fascinating. How much training do you have in the technical aspects of AI? You obviously learned a lot along the way.
Dragoş
Zero!
Tom
But do you need to be a coder to legislate on code?
Dragoş
I'm a lawyer. I don't have a technical background, and I barely use my phone. And certainly I use it less than worse than my kids do. But I grew interested in the topic mostly in its implications, the more political and strategic implications of artificial intelligence. That's how I decided to focus on this topic during this mandate. And in the process of doing it, I have educated myself to the point of being able to legislate. But I remain a legislator, and my knowledge of AI implies all of its implications rather than the intrinsic workings of an algorithm or how you write it.
Tom
In a sense, that could be an advantage. However, if you're looking at it from a humane – a human perspective, and a humane perspective, I've talked with coders at the Pentagon who don't understand the question of self-targeting killer drones being part of their responsibility. They're like, “No, no, I just write the code.” Actually. . . .
Dragoş
Exactly! No no, I agree with you. I actually think a certain distance from the technology itself and how it works, I think it helps to keep things in perspective, without getting that bias, speaking of biases, that bias of being too much into it, too much connected to how it actually is being developed, designed and rolled out. Therefore, you might see it as something that is just that just a piece of technology. And I hear that argument a lot: It's just a piece of technology. It's just an algorithm. What can do what harm it can do? And am I responsible for the harm? No, I mean, I just put it there, up on the market. And of course, you can understand where they come from, for their argument, but that is a responsibility that we have, as policymakers, to think past that, to think of the implications, and then again, put the perspectives in place to come to the right decisions as to how we need to regulate or put guardrails in place to make sure that the technology stays safe.
Tom
Well, the AI Act is a magnificent piece of writing, and of thinking. And I encourage everyone who's listening to this to, to pick up a copy and read it carefully, because it is one of the key, foundational documents for our future. Thank you for working on that.
Dragoş
Subjective, of course, but I do think it's an important piece of legislation that we have produced during this time.
Tom
Objectivity is a very elusive goal in this activity, is it not? I'm going to jump over a lot of questions that I had for you because that was that was a 15 minute question right there. We could spend an entire podcast on that, and not scratch the surface.
But can you discuss perhaps, moving on to security and defense, a few of the examples of how AI is needed to recognize trends, objects, risks and do better things that customs and law enforcement already do, while of course remaining, as we've said, cautious and fully transparent.
Dragoş
Well, I won’t repeat what I said earlier about criminals being always ahead of the game when it comes to technology. But particularly as a lot of our crime has moved online, in the last, I would say, even 20 years by now -- time flies -- and therefore, clearly the tools to fight either entire criminal activities taking place online, to parts of criminal activities are taking place online, particularly money trails, or financial transactions that are linked, or are the consequence of criminal activity. So for all, that law enforcement needs similar tools, or even more sophisticated tools, to be able to detect or to counteract or to respond to them. And if we don't understand that it is absolutely paramount that we equip our law enforcement with those tools, and if we somehow remain in the logic or the narrative that all tools in the hands of law enforcement can lead to abuses -- yes, they can lead to abuses, which is why you need safeguards in place to make sure that they don't do that, but at the same time, not seeing the need to adopt, and to encourage, in fact law enforcement to also adopt new technologies, in this case, AI, I think is a very short-sighted understanding of what we need to do for our security.
The same goes for defense. In a way, the war -- not “in a way”, clearly the war in Ukraine has opened our eyes as to what are the new realities of kinetic warfare. Weapon systems that we all believed were absolutely essential in a war, or that had absolutely no response in warfare, the reality has proven very differently. And in fact, we now see more and more how even a full-fledged war, like the one in Ukraine, relies on new technologies much more than we anticipated. And in fact, it seems like we're heading in a new era where defense and warfare is going to be based on attritable weapon systems that are not very expensive in terms of producing them, but that can be very effective in destroying equipment that are worth hundreds of millions of dollars.
So, what does that mean? That means that because all of these attritable weapons systems, they need a brain, a smaller or bigger brain, because most of them are autonomous in their function. All sorts of drones, from maritime, to air to land drones. And as is I think evident by now, that is the future, the direction of travel, in terms of the future in defense. And this is where, of course, fundamental questions are popping up. They're already popping up in various war games that are being organised by NATO, either collectively or individually by NATO members. And I'm sure the same is happening in other corners of the world. How far can you go in developing these autonomous brains of attritable weapon systems to let them take decisions in the middle of a war? Or even bigger, if you run into war scenarios, on the basis of assessments that are being done by algorithms, what risks come with that? What does that imply?
And I've just read yesterday, a piece of interesting news. One of the latest such war games that were being planned and rolled out in the US, showed how AI that was being asked to develop scenarios all the way to the end, in response to hypothetical threats, tended to be much more violent in the type of responses that they were proposing, leading all the way to the kind of response that I think any human being would have a very hard time picking a choice of action. So that shows that again, over-reliance on technology in the area of defense, particularly when it comes to those final moments when you make recommendations or decisions that are leading to certain courses of action. That is something that has to be taken with a lot of grains of salt, with a lot of care. Because again, it's certainly not infallible, and also, it's certainly not desirable that we work on the only on the basis of this scenario. So certainly, again, it's a fact: we have to also in the area of security and defense embark on this journey of the digital transformation. That's I think beyond doubt now. But how we do that, how we make this integration, keeping always the human in the loop? It's the same principles that we have, for example, in the AI Act, where we say that for certain types of recommendations and decisions there always needs to be a human in the loop, in order to keep that element of safety in those recommendations or decisions that are being made by an algorithm. The same, or even more so, need to be true in the area of security and defense employing these technologies.
Tom
Well, that's a sobering scenario. Yes, without the human in the loop, you don't have the safeguard of saying, “Wait a second!” The “sanity check,” as it were.
Dragoş
Absolutely.
Tom
Well, moving on to international affairs and transatlantic issues on foreign affairs, do you see customs policy as a tool to support foreign policy?
Dragoş
Well, of course, it is. Maybe it's a bit unseen by the naked eye. You don’t automatically or easily think of customs when you when you speak of international relations. But I think in the way customs works, in the fluidity of the movement of merchandise, of trade, across borders -- more generally, the economic system as we have it at the global stage relies very much on the way customs functions, or doesn't. So in a way, a good and well-run customs system adds and helps how the relations between states or blocks are evolving. So yes, there is a link.
Tom
And on migration, do you see a closer cooperation between border forces as useful or even necessary? I'm thinking of customs and Frontex cooperation, that sort of thing.
Dragoş
In international relations, or confined to the EU? Because either way, again, it's valid for customs and product is certainly valid for people-moving. Or even more so, because migration has been a phenomenon that has been so much and so prominently a part of the debate, societal, as well as political debate, pretty much everywhere, not only in Europe, but also in the US. And we see how much it has actually been used to poison society and politics. That, number one, certainly it is relevant. So migration is a key political topic. Not only within a political space such as the EU, or in a political space such as the US, or in other parts of the world; it is also valid for how international relations and the diplomatic dialogue goes.
Solutions that are being adopted for enhancing border security, and those agencies that are responsible for it -- in our case, Frontex and the national border authorities -- are also key. And, again, if we only look at the last two political cycles in the EU, starting with 2013-2014, when the migration the first wave in what has become almost a permanent migration crisis in Europe, the issue of border security has always been at the very center of the debate. How to do it, how to enforce it, what kind of extra tools do you need, what kind of extra resources do you need, how much European coordination versus national competencies and responsibilities? So all of these are still ongoing, and I don't think there is going to be an end to this debate anytime soon.
And as far as we're concerned, here in the Union, it’s also essential for keeping the Schengen Area going. Without safe borders, you can't really have reliable system of free circulation within the union. So it's even more of a reason to make the right investments, and make sure that that actually works.
Tom
And you saw the first migration crisis very up close and personal, didn't you?
Dragoş
I saw it very up close and personal. I was in the heart of it, in a way, because I was responsible at the time for the strategy and coordination unit of DG Home Affairs and the Commission. And in fact, we prepared the first responses to the migration crisis. And then I moved on to government, where as Minister of Interior, in the Council, we worked on trying to find the right set of answers, 2015-2016, to the migration crisis. And we couldn't. I mean, we found some answers, not all the way as we wanted it, and the files that blocked politically. In fact, only now, after all this time, in December, we also found a political agreement -- let's see how that passes Council and Parliament -- on the migration pact, which is meant to finally try and put an end to a very long cycle of trying to amend our migration policies at the EU level.
Tom
Yes. And in the meantime, both here on this side of the Atlantic and the other side of the Atlantic, the Right makes significant headway and significant ammunition out of this issue -- as you say, poisoning the debate in many cases.
Dragoş
Of course, it has been it's a topic that has been politically weaponized, certainly by the extreme right, but not only. And, as a researcher once told me, someone who has researched all migration crisis both in the US and the EU over the last 100 years, his conclusion was that, in fact, the only times when governments made the right decisions about migration were when no one was paying attention to migration. In other words, when it was not part of the political debate. Because when it is part of the political debate, usually the solutions are not necessarily the wisest.
Tom
Wow, amazing. The other is always a threat, a boogeyman that can be used. I need to be respectful of your time Dragoş. So I will jump to closing questions. PEN-CP, the EU Horizon Project that runs this podcast aims to accelerate technological innovation in customs. Considering your strong involvement in fostering new technologies, how would you suggest Customs Administrations strengthen their collaborative ties with researchers and universities and startup companies?
Dragoş
Well, I think it's essential. Again, because of how omnipresent AI and generally these new technologies are going to be in the years to come, I think anyone that remains behind in adopting and in reaching out towards those that actually hold the keys, to a large extent, to this technology -- again, anyone that does not understand the urgency of that I think is going to lose in what is a competition, what is the race. And what is specific to this new, let's call it “industrial revolution,” is that a lot of the means are not in the hands of the authorities. The data is not in the hands of the authorities, the talent is not in the hands of the authorities, the infrastructure, whether it's the computer power or the networks used, or the cloud, almost all of these means are, in fact, in the hands of private sector or research communities. And I think because of that, public authorities -- in this case customs authorities or border authorities -- it's not a choice, or a luxury. It's an absolute must that they reach out to the business community, they reach out to the research community, because it is the only way to actually make the right decisions and adopt the right technology. Because that's where, again, the talent is, that's where the data is. That's where the technology is.
Tom
And at the same time, it's in the best interest of these individual private sector players to have a safe border to have a sound, sort of . . . .
Dragoş
Absolutely, absolutely. And they stand to gain from public authorities uptaking this technology and integrating it in processes that are also ultimately beneficial to them as well.
Tom
Well put. Given that PEN-CP is funded by the EU and given that you also know DG Home policies very well. What is your view of the importance of investing in European Research and Innovation for security now and in the future?
Dragoş
I think it's crucial that that is being done. I'm happy to see and in fact, I also participated actively as a shadow Rapporteur on the upgrade to the mandate of Europol, for example. I was also part of the scrutiny group for Frontex operations and Frontex evolution. And I think it's key that they remain -- not only they remain open, but they have a proactive strategy in developing their own capabilities growing their own ability to not only use but even, why not? develop technologies and solutions of their own, which they need to employ to the benefit of their “customers,” whether they are the law enforcement authorities of member states or the border authorities of member states. And the Europol has set up this lab, in which there is funding also from the EU side. And also a lot of investment to bring the right expertise in there. And I think it's the way to go.
Tom
So for you EU Horizon grants are an important part of the future changing landscape?
Dragoş
Absolutely.
Tom
Last question, if I may, if I may ask you, are you what are your plans for the upcoming elections? Are you running for re-election?
Dragoş
I haven't decided yet. I still have two weeks before I make up my mind. I have several options on the table. So therefore I have to decide which one I take. I'll think of how I best continue the things that I have done so far, and I'll make my decision then.
Tom
Well, it doesn't surprise me that you have many options. But based on the quality of this conversation, I beg you to stay where you are! Please run for re- election. Thank you so much for joining me, this has been a fascinating conversation. I really appreciate it.
Dragoş
You’re very welcome! Goodbye.
Tom
Thank you very much, Dragoş Tudorache, member of the European Parliament, for sharing your insights on the past and future evolution of customs, from the point of view of Europe of a European legislator, an expert on AI, security, defense and more.
You've been listening to “Pushing the Envelope: Life at the Cutting Edge of Customs Innovation,” a podcasts that profiles leaders and customs, and their views, ideas and inspirations about the future of customs and postal innovation. Join us soon for our next podcast and another insightful conversation with customs trendsetters and forward thinkers. Thank you!
Materials and further reading on the AI Act:
- AI Act – overview: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
- AI Act – texts: https://artificialintelligenceact.eu/the-act/
- AI Act – full set of documents: https://artificialintelligenceact.eu/documents/
- A few select analyses of key aspects of the Act:
- Recent studies show wargaming with AI results in more violent choices:
o https://techreport.com/news/research-shows-that-ai-tends-to-make-more-violent-choices-in-war-games/