Plenary RIPE 84
.
16 May 2022
.
2 p.m.
MIRJAM KUHNE: I was just going to say, please take your seats in the back, there are still some seats on the outer parts of the room. This is a bit of an awkward layout from here, I can't see you all. And then we can get this show started.
Well, welcome to RIPE 84.
(Applause)
.
Yeah yeah yeah, real people!!!
.
My name is Mirjam Kuhne, I am the Chair of the RIPE community and this is actual my first meeting I am chairing in person, and this is the first face‑to‑face meeting we are having since RIPE 79 in Rotterdam, I think, was it May 2020? I don't know. October ‑‑ well a long time ago since then we have only had online meetings and we have actually learnt a lot during those online sessions, and I hope ‑‑ well, I'll get back to that a bit more on the how we tried to incorporate the online participants as well, but I wanted to introduce you also to the RIPE Vice‑Chair Niall O'Reilly, who also started with me in September 2020 and has only done things online, so we have actually only met yesterday for the first time ‑ I mean, for the first time since we started as the RIPE Chair team, so we have done everything online too.
NIALL O'REILLY: I am really excited to be here in person again after so long, and it's great to be working with Mirjam, so, welcome from me too to RIPE 84.
.
(Applause)
MIRJAM KUHNE: We have an amazing number of 1,63 registrations in total, which is bigger than I had expected, to be honest, and we had like 685 online reg ‑‑ on‑site registrations, of which have now checked in already 453, so, there will probably still be more coming, I hope we are all going to fit into this room here, it looks a bit cramped already but I think you are leaving enough space, then we will have also over 400 online registrations, which we have now 388 checked in and probably watching us now online and participating online.
So, as I said, we have learned a lot during the pandemic and during the online events and we have really tried to put together a truly hybrid meeting and so for the RIPE NCC it's been double work to organise an onsite and an online meeting in parallel, we are trying our best to also make the online participants feel included, that's why we are also using the Meetecho platform still during this week so that people online can participate and ask questions and we'll see how it goes, it's a bit of a challenge for the session Chairs to keep track of the queue, which we'll try to make it one queue online and onsite participations.
You can see here also how many countries, people from 62 countries have registered for this meeting so it's quite a diverse meeting in terms of nationalities, which is great.
.
And on top of all that I am actually from Berlin, so it's really cool to be back in my home town, birth town, I remember there was another RIPE meeting in Berlin here kind of across the road many years ago, we were a lot smaller then, so, yeah, it's a lot coming together, you know, seeing all these people face‑to‑face and being back in Berlin and, as I say, really exciting.
So ‑‑ this is the first time I am doing this in such a large room, so give me a minute, it still feels a bit uncomfortable to be here with all of you together in the room instead of just having a big screen in front of me.
So that's actual one thing I wanted to point out. It's fantastic we are all here together. There are still a few things going on, I think, that, you know, at least keep me awake at night, so maybe I can say a few words about that.
First of all, I really have to learn or relearn how to socialise again with people. I want to hug you all but I am still a bit careful because of Covid, so feels a bit uncomfortable. Second thing, obviously Covid is still among us. I had another community meeting last week where a lot of people got sick in the end. I hope that's not going to happen here, so I would like to, you know, urge us all to be gentle, tolerant with each other, respect each other's choices, you know, when it comes to distancing or wearing masks. I am still wearing a mask also because I feel more safer for myself but also as a sign don't feel bad if you feel more comfortable.
So there are still people wearing masks and it's totally fine. We are handing them out here at the registration desk, they are also out in the hall and also we are handing out self‑tests. I think they can also be ‑‑ you can pick them up at the registration desk if you want to do a test before you come down to the meeting. And I would like you to ask you to take your responsibility for yourself and for other people's health as well.
And then, the last thing, another big thing that kind of keeps me awake at night is, of course, the war that's going on in our region here, and then I'm sure we'll have people here from both Ukraine and Russia, and, you know, we are all part of this one community, and we are colleagues and I hope things will go well. And, you know, we might have some discussions and I'm sure you'll hear more about it. I see a lot of presentations on the agenda this time that refer back to that war.
I also want to stress that not only have you kept this Internet running during the pandemic, which I still find amazing and I still like to applaud all the network operators who made that happen and kind of in the background, the world was kind of relying on you all but now during the war it's amazing out the operators on the ground in these difficult circumstances keep the Internet running up and running in the Ukraine, but I think it's also a fascinating to see that the infrastructure in Ukraine is really very decentralised and that's really, I found, you know, maybe one of the reasons why it's still up and running and I think you also hear more about that later today.
.
(Applause)
.
Applause to all the operators out there trying to keep the network running.
.
I have a slide here ‑‑ wait, I had a slide here with the Code of Conduct, but that's disappeared, maybe we'll move that to another place.
Now, actually, this is a good time, because I wanted to pause here and give the word to Vesna, who wanted to say a few words to you, she is the long‑term community participant and she is also the senior community builder at the RIPE NCC, she is still stuck on the train, maybe she has already made it to Berlin and she is on the bus and she can't be here, I'll do this for you.
She has done this a few times at other meetings and I thought it was powerful and brought the community together so I'm trying to do a Vesna here with me. She is a lot more natural than me.
.
It's about a minute of silence.
"I am inviting you to join me in acknowledging all our losses since the last time we have met. We have lost lives, health, connections due to war, climate catastrophes and the pandemic. Let us honour our grief, mourn for those who have died, be sad for all the suffering and experiences ourselves, our families and friends and our communities. You are not the only one who feels this way, you are not alone. And we are all in this together. So let's join in a moment of silence as a vessel to hold our sorrow. So I would like to actually all let you all get up, if possible, and keep a minute of silence and memorising ‑‑ remembering all this.
(Minute's silence)
Thank you all. Now we can move on.
We have gave this a place, and may be you kind to yourself and may you show kindness to each other and thanks to Vesna for preparing this and sorry she can't be here, she might pop in any moment now.
(Applause)
.
One of my biggest worries during the pandemic also was that we kind of die out as a community. Everything is ‑‑ well not literally, sorry, that maybe was a bit harsh after this minute of silence. I mean as ‑‑ because we are so dependent on our human interaction with each other and we are based on trust and social interaction and we just couldn't do this in the last two years and maybe those of who who met before and have known each other for a long time it was a little easier because we could build on that, but it was hard for new people I think to enter. So I'm really happy to see a lot of new faces here, we had the newcomers' session earlier today. Many people are here for the first time, which is great, and I hope they all come back and you all be nice to them and include them in the community.
.
And I wanted to say something about the social interaction. That's why I'm bringing this up. We have prepared also some social interaction for online participants in the hall so that they also feel included and I know we all are happy to meet each other and talk to each other again but we shouldn't forget our online participants as well.
Now, let's look a bit more into detail. I don't know if you can read this. This is a bit ugly.
That's the meeting plan. You have all seen it, it's on your badge, I want to point out a few unusual things or a few things that are maybe out of the ordinary.
.
You see we have two BoFs in the afternoon today and tomorrow. Today the BoF is about what we want from networking proposals in the future. It's kind of a wish‑list of today's and future Internet, so more like technical BoF. And tomorrow the RIPE Code of Conduct Task Force is meeting to talk about the recruiting process for the RIPE Code of Conduct team in the future. And also still gathering feedback on the process and reporting for the Code of Conduct that they have sent out to the community. So they are still gathering feedback on that.
Tomorrow we are also have for the first time the best current operational practices task force meet again, and they have a really interesting agenda, also again linking back to a sustainable and reliable Internet in times of crises and wars and catastrophes and I think another topic is about sustainable networking, as in like environment al sustainable mostly. So they will be exciting.
You can see today and tomorrow, as usual there is Plenary talks. Tomorrow we have also ‑‑ I don't know if I have a slide on this, we might, I'll show you in a second, we have a Women in Tech session again, I am really excited about this because we didn't have those during the pandemic because we didn't want to have, you know, things during the breaks and it felt unnatural but this is really exciting this session. Let me see if I can find the slide.
.
Maybe I shouldn't have done this. Go back. Ah, well, just ignore all this. We are going to remove it from the reporting to ‑‑ go back to the meeting, this is the first time I am doing this in real life.
.
Anyway we'll have a Women in Tech session with some really interesting topics. We are going to talk about the gender data gap and what that means, we'll have Shane talk about the measurements he has been doing over the years like diversity in this community and what we can learn and how we can improve it in the future.
Then as usual, Wednesday and Thursday we'll have Working Group sessions, the RIPE NCC General Meeting in the evening and then on ‑‑ on Wednesday, and then on Friday morning we'll have Plenary talks again and a Closing Plenary.
.
This is the Programme Committee. Some of them are here. Some couldn't make it and these are the people that put together the agenda for the Plenary for you, and they have done a fantastic job I think, it's a really interesting programme, but they of course always welcome feedback from you, please rate the talks during the week on the website.
And we'll also have elections going on again. Two seats will become available on the PC this time around, you can still put in your nominations until tomorrow, and then voting will take place until Thursday and the results will be announced on Friday morning.
.
Sorry, I cannot ‑‑ I have the feeling that I am ignoring you over there because it's such a wide room it's kind of hard to see everyone at the same time.
.
This is a list of all the Working Group Chairs we have at the moment, again not everybody could be here. There are a few new people also, new kids on the block that have been selected in the meantime, and they will start the new, you know, term at this meeting. You can see on each Working Group's page, these are also who the Working Group Chairs are and when the next nomination comes around. You will see many of those during the week and they are the ones putting together the agendas for the Working Groups, together with the Working Group of course.
.
(Applause)
.
They are volunteers, they are doing this in their spare time.
.
Now, most important part maybe, social events at the meeting:
.
There is a welcome reception today, as usual, Monday, and after the Plenary ‑‑ after everything finishes today here in the hotel, and there is a networking event tomorrow, and then on Thursday we'll have the traditional RIPE dinner again in the real world. So, it's going to be exciting. I know the venue, it's really quite a nice beautiful like traditional Berlin style venue, hall.
.
We also have some events for, specifically for online participants. They can participate in the welcome reception and then we also prepared a small online RIPE dinner. They were quite popular during the pandemic and so we thought we were going to continue that and we are going to use again the SpatialChat social platform that we have been using during the online events, and there are some NCC staff out there at the coffee area trying to kind of engage a bit more between like ‑‑ link between the onsite and the online participants.
.
This is our Code of Conduct. I already mentioned the Code of Conduct Task Force. They, together with the community, have worked on a new and improved Code of Conduct that came in place after the ‑‑ at the last RIPE meeting, I believe, and so this is not everything. This is just a small snippet, but you should have, you know, seen it all when you registered and it's on the website, on the RIPE 84 website but also on the generic ripe.net website, so I'd like to again ask you to treat each other with respect and tolerance and this is ‑‑
.
Finally, these are the trusted contacts in case there is something about this that you feel uncomfortable or you feel mistreated. You can reach out to the trusted contacts. They are trained staff, not only staff, three RIPE NCC staff, one community member, Rob Evans over there, and there are a few more people and there is the e‑mail address and everything is dealt with confidentially.
And last but not least, I would like to give a big applause to our local host, DE‑CIX, and all the sponsors who have made this meeting, helped us to put the meeting together here in Berlin.
(Applause)
.
And I would also like to thank the staff of DNOG and BCIX who put together a fantastic pre‑social event last night, I thought it was a really good initiative.
(Applause)
.
And I think that's it. That's us. This is us. If you want to reach us during the week, we are around, Niall and I, and I'll hand it back to Jan and Franziska.
No questions.
.
THOMAS KING: So, slides are coming up. Perfect.
.
So, also a warm welcome from my side. My name is Thomas King, I am the CTO of DE‑CIX and it's great to host a RIPE meeting again in Berlin.
.
It's actually the third time that we meet here in Berlin for a RIPE meeting and it's the second time that DE‑CIX is hosting a RIPE meeting in Berlin, and we did it the last time in 2008, and, when I prepared for this presentation, I actually looked up some of the pictures which we took in 2008, and you might remember some of the people from back then, because some of them are in the room today, which is great, right. But what is even better from my point of view is that we have so many new people in this room and that the community was growing over the last 13 years like crazy, and that's great to see.
.
And that's only one example of how important it is what we do as a community for the society. And I want to give you another example, and for that, let us quickly highlight how DE‑CIX was looking into 2008, because back then, we were mainly providing peering as a product, and we had only operations in Frankfurt, Hamburg and Munich and we were pushing about 500 gigs of traffic, you know, back then it was a lot of traffic, nowadays we push about 15 per bits of traffic in 34 areas all over the world.
.
And this shows you, you know, how far we have come in the last 13 years because now of course we still provide peering but also Cloud connectivity, closed groups and security fears like blackholing advanced to different target groups like wholesale and also enterprises. And, you know, what is the reason for this development?
.
And you all know that, digital is now everywhere. Look into the cars, the displays get bigger and bigger because we all want to consume digital services while driving. We have learned during the pandemic that we can run a whole company just by jumping on Zoom calls all day. And in all the other areas of, you know, of what we do in our life, there is digital, aad digital is for everywhere, for everything and everyone, and I believe we are just seeing the beginning of what the journey will be.
.
And especially, if you look at the inter‑connection industry, we see that latency is the new currency because now content needs to be very close to the users and, you know, to make all this nice applications like Cloud video‑gaming and streaming and virtual reality a reality, this is why it's all around.
.
And, you know, I hope in 13 years down the road we will meet again here in Berlin and we can talk about how great our future was, you know, looking back then, because I believe we have a very bright future ahead of us so I hope you enjoy this day here in Berlin and you have a very successful RIPE meeting.
.
Thank you very much.
(Applause)
.
HANS PETTER HOLEN: So, welcome everyone. My name is Hans Petter Holen and I have already seen on Facebook that somebody thought I was still the RIPE Chair, but no, I am not. Two years ago I started as Managing Director of the RIPE NCC, and I was volunteered here today to give some meeting logistics on behalf of our meeting team.
.
So, Meetecho is the main platform for this meeting. It requires registration, bookmark your unique link and use it for all sessions. And you can do questions in both the Q&A and using audio and there is a live transcript available, just as you can see on the monitors here.
There is also a live‑stream page, so if you don't want to log in and be an active participant you can watch the whole performance online.
.
The sessions are already recorded, so if you fall asleep during the sessions you can watch them later if you want to do that.
.
In Meetecho, there are different buttons for audio and video when you want to ask questions there is a Q&A icon where you can type your question, please remember your affiliation, there is stenography here, and, as you probably know, they write what I'm saying on the screen behind me, or what I should have been saying; I never know what comes up there.
.
There is a chat group so you can interact one to one and a poll for questions for polling during the talks, and also a participant list.
.
If you want to participate on site, we have brand new meeting application. So it's a web application that you can go to, it's a link in an e‑mail you have received, one for each of the rooms where you can participate easily in the chat and you can raise your hand in order to ask questions. Now, in order to ease the logistics, please then also go to the microphone but then in order to enter the queue you need to raise your hand in this new fancy app so that we get one queue and don't prioritise people online or offline.
.
Network ‑ this is not about shifting IP packets but this is about talking to each other. We have SpatialChat. So it's a virtual arena that you have used if you have been to our virtual meetings during the pandemic. It's open all day. And you can use the link and password to participate in that.
.
There is also the networking app that we have had for several meetings where you can schedule meetings with fellow attendees, and the attendee list on the RIPE website.
.
Covid:
.
Covid is not quite gone yet so please be careful, please test each morning, we have tests available at the registration desk. Please stay in your room if you test positive. Please wear face masks if you feel that's appropriate for you. There are no hard rules to do that, but it's quite okay to do so, as Mirjam already said. Masks are, however, mandatory on public transport and on the shuttle buses to social events, so please remember that.
.
And we have in the room here added extra distance between the seats so that you should feel a bit safer than you have on previous meetings.
.
More information is on the website. And we we are really happy to have this in‑person event again, so let's do our best and keep each other safe.
.
There are also some stickers on the ‑‑ available that you can put on your badge to indicate whether you still prefer and would like people to observe distancing, use the red one; yellow, elbow bumps are okay; or green ones for hugs and handshakes. I was looking for the black one to signify the Nordic 5‑metre distancing but we have missed out on that one.
.
Anyway, thanks for coming and good luck with the meetings, and then back to the meeting Chairs.
.
(Applause)
.
CHAIR: Okay. Welcome also from the Programme Committee side. This is now officially taken over, this session is now officially taken over by the Programme Committee, chaired by me, I am Franziska and this is Jan, and we start with a blog on a topic that is on all of our minds these days. With that, I give you Emile Aben from the RIPE NCC, who will present us with measurements on the resilience of the Ukraine Internet. Welcome.
EMILE ABEN: I hope you can all hear me, I am not on mute.
So, hi people that care about the Internet. I am Emile. I work for the RIPE NCC and I want to share some observations that we did on the Internet in Ukraine.
.
It's great that we can be here. It's been two‑and‑a‑half years, and, in the meantime, we had this pandemic and then there was another thing that kept a lot of us in the minds of a lot of us I think, it's in my mind at least, and that's the Internet in Ukraine, or Ukraine, of course, but as people who care about the Internet, we of course also care about what's going on in Ukraine, specifically for the Internet there.
.
And I have been obsessively looking at the news and at measurements since the war there started. And it was actually my expectation and maybe a lot of you had the same expectation that the Internet in Ukraine would actually not hold up very well, given all of the things going on there. I expected a rapid deterioration of Internet conditions there, cyber warfare capabilities and all such, but the reality, at least in as far as you can see in measurement data, it's still functioning, it's damaged but still functioning, and I think the best indication of that is a project that is visible in this project called Ioda, and the picture is here and it this gives you a 90‑day overview because we are already 90 days into this. And what you see here is that things have kept mostly on a bigger scale ‑‑ have been online, working. If you see there is like 100% line at the top, that's like the pre‑war conditions pre‑February 28th, and you see the green line, for instance, is BGP, that's like the number of /24 equivalents visible in the global table and that's basically, it's a little downward trend there in the beginning, but mostly it's been at a stable level.
.
If you look at the blue line here, that's like IP addresses that are seen active, so they are seen responding, that's actually gone down a little bit. It's like if you look ‑‑ it's at 80, 85% of the pre‑war levels.
.
And that can, of course, be explained by damage, war damage. But there is also an alternative explanation here in that a lot of Ukrainians have left their homes, or are displaced, in a country of 44 million, millions have been on the move, fleeing, and what ‑‑ that could have also caused an effect of people just shutting down equipment, like their home or office, and that could also explain a little bit of that effect.
.
The orange here is systems that do like Internet background radiation, basically, and that's a little bit more erratic signal. We see that go down but it's less clear what that means.
.
Going to a measurement platform that I know a little bit better ‑ RIPE Atlas. Of course we have, like, 11,000 RIPE Atlas probes deployed worldwide, thanks to many of you in the room here, and also, we have them in Ukraine. Before the war we had 220, 225, and we see the same slow downward trend in the beginning and it's being kept stable for weeks now. We actually made an online interface where you can see the number of RIPE Atlas probes connected for a particular country, for a particular autonomous system, or for a particular city with a radius, and you can visit that on the URL that's on this slide.
.
And that's actually an open sourcey thing that you can modify yourself, use it in ‑‑ however you want, and I invite people to actually use that. You can look at other events with this.
.
You also see little downward, like ‑‑ little downward spikes which actually means multiple probes at the same time go down, which typically is a signal something is actually going on. And I want to Zoom into a specific event that we saw. It's the network called Triolan AS13188, and you see can see in the same interface, you can see that we had 11 probes there and it went down 3 probes all at the same time, and then recovering in two days. This is an interesting case for a couple of reasons. One is that the Triolan people documented what happened here, and this was, according to the documentation they put on their website, this was an act of sabotage. So there were attackers getting admin passwords and then just causing chaos in their network, and it's actually amazing that they got this back up in two days again.
.
The other thing is that like this is, well, quite a dramatic drop, but if you look at it in the big picture, it's roughly 2% of users in Ukraine use this network, so, on the bigger ‑‑ if you zoom out, it was not a huge amount of damage done by this network being down.
.
So, but this is actually one of the examples of damage that happened there. There is other cases like this, but as you can see, things go up again, things ‑‑ people made things function again. But let me get to the worst thing that is going on there currently. That's the situation in Mariupol, described by the Red Cross as this topic. Things are really bad there, and of course these are the circumstances where the Internet breaks, there is no power, no fibre, and that basically makes that thing stop. The Internet is a human‑made system so it breaks at a certain point.
.
And a case of that is this, the Illich iron and steel works, they have their own AS and it's located in, Mariupol and what you can actually see in the data collected is, at the beginning of the war, they were still online, they had two networks they depended on. You can see there is two ‑‑ a red line and a green line here, that are their dependencies. After a little while, one of them was left. So March 2, you see there is only the green line, which is their major dependency and, after that, they just dropped out of the global table.
.
So, there is a breaking point to this.
.
But with that, if you zoom back out, what I said at the beginning, things mostly kept functioning, and that was like, I was surprised. I spoke with others that were quite surprised by this. So, why did it not break more massively? And we looked in this, there is a couple of layers that we discovered in the data.
.
One of the things, a very important thing there is not in our data, and that's people like this ‑‑ well, repairing your network in these types of situations, for me, that was a massive amount of respect. And this is a very important part ‑‑ people just keep working to just keep this connected, and this is amazing work being done by people on the ground there.
.
So, it's humans, dedication, courage, actual real people like you in the room here doing this type of work.
What we had in our data is other signals of why things might be ‑‑ maybe kept working, and one of them is ISP decentralisation, so there is not really a very dominant market player here. It's quite decentralised. There is diverse inter‑connections, there is lots of IXPs we can see in measurement data.
.
Another thing is high independence at a transit level and we have the IHR, the Internet health report. There is a link here that can actually show you what networks Ukraine depends on for its connectivity. And tele geography had a nice article about the diversity of fibre path, the physical diversity but also multiple organisations, so you see multiple directions and, in all of them, you have this redundancy, this resiliency. And let me focus on two in more detail.
.
The first one is the ISP decentralisation, and now I have to ‑‑ Herfindahl–Hirschman Index, HHI ‑‑ something that comes with age that you cannot remember these terms any more.
.
This is an index used by ‑‑ in economics and it's a scale from 0 to 1 where 1 means an absolute monopoly and 0 is like a totally disperse market, very small ‑‑ lots of very small players. And if you just calculate this based on data that APNIC is providing, and we have this data ‑‑ there is a GitHub repository online that you can find through the slides ‑‑ if you look at this data, Ukraine is fourth in the world in lowest HHI. So it's very dispersed. And we also have a visualisation of that, and what you see here is ‑‑ this is a full circle, the full circle represents all of the end users in Ukraine, but for this visualisation, we chose to only show the networks that have over 1% of market share. So you can actually see 55% of them is not shown, which actually shows you how dispersed the market is and what we show here actually is the elements that interconnect these networks. We measured this with RIPE Atlas and you also see a lot of elements there that do this for the main networks and the orange ones here are the IXPs, and the blue ones are other networks.
.
There is a couple of links there. This is measured with RIPE Atlas and there is some visualisations on that also open.
.
The other thing that we looked at is IXP abundance. You see, if you look at PeeringDB, there is 19 IXPs in Ukraine, and if you just measure between all the Atlas probes you see 13 of them we actually see between the probes. And the other thing here is that if you just look at the colours, each colour is a different IXP. There is not really a dominant colour, so there is not a single dominant IXP, it's just like a very dispersed, which helps in resiliency.
Now, I'll take a brief detour, what happens in occupied territories.
.
This is an interesting thing we saw happening around May 4th in Kherson, and Kherson is a town in the south, it's relatively close to Crimea, and what you actually see is ‑‑ maybe it's not very visible ‑‑ it's too small ‑‑ you see the dominant ISPs here are mainland Ukrainian and it switched over to using a network called Miranda Media and that network is very well known, documented that that provides Internet connectivity for Crimea.
.
So this is sort of like a rerouting of traffic instead of going this way, it now goes this way.
.
They switched back, but it's a signal of what's actually going on in that town right now. And it's part of a larger thing that has been happening, and this is a study that I want to briefly touch upon about the ‑‑ what we called an Internet interregnum, an unstable period in between stable periods, which happened 2014, 2017, and in 2014 Crimea was annexed by Russia and we took a deep dive and we studied that together with ‑‑ she is a sociologist and she did some field work in Crimea and what came out of that actually is a lot of like evidence of people in criminal working together, both the Ukrainian and the Russian side, to make this Internet work.
And we used something called AS Hegemony, that's like how dependent is a network dependent on other networks. There is a link there so you can look at that for your own network we found some BGP data that is in RIPE RIS.
What you see here is that, like, the network went from say Ukrainian influence to Russian influence, not in a single event but this actually took three years for it to completely move over. And this is sort of like an indication of that.
.
In the top here, the colours indicate the ‑‑ how dependent a network is ‑‑ how dependent a set of networks is on another network. So, the darker, the more dependent. And this is the timeline. So in the top you can see the Ukrainian networks, they depend on a particular set of networks and that doesn't change that much. Russia, it's the same. But if you zoom into Crimea specifically, you can actually see changes happening from 2014 up until 2018, with like this is Rostelecom and Miranda Media really taking a dominant position in this.
.
So it's like significant changes through the connectivity there, it was a long transition and it was a good match, the Internet data was a good match with a timeline that we got from the interviews with the locals actually. And it's a topological chokepoint reflecting geopolitics in the region.
.
So, with that, I come to conclusions here.
.
Takeaways: We tried to build to try to get some or to provide insight on top of our data, apart from all of the established interface that is we have, RIPE RIS, RIPE Atlas and RIPEstat on top of that, we also build some prototypey things. This is open source, it's open to tinker with and to improve, and it's built on top of Atlas and RIS, with all of your help.
And main takeaways here: I think Ukrainian Internet seems to be a very resilient Internet and we see many redundancies in many layers, but it remains a human‑made system that has its breaking points.
This is not the most important part of my talk. The most important part of my talk is actually this, and it's actually the next talk, so, I am not going to spill a lot about this, but this is the really important work that's being done, and that's what the next talk is going to be about. So... thank you for your attention.
.
(Applause)
CHAIR: Thank you, Emile. Thank you very much for this. Now we have Q&A. Is anyone in the queue? So this is an experiment now. Everybody in the room needs to ‑‑ you need an app that needs to go into the queue and then we will call your name to come on the mic and ‑‑ or you can just come up to the microphone
RUDIGER VOLK: Thank you. At IETF, I worked out that I don't need an app.
Sorry, Rudiger Volk, no affiliation, but just myself these days.
Emile, one question I have is: Some publicity was around by the ‑‑ beyond oligarch‑rich people supplying support for Ukraine Internet. Did you see anything about that?
EMILE ABEN: I did not see ‑‑ I'm guessing you are talking about Starlink?
RUDIGER VOLK: Yeah, sure. No oligarch is actually marching Elon Musk, we know.
EMILE ABEN: Yeah. There was a lot of publicity around that and I was, for a while I was like I share your scepticism that I sense, but then again I say, for instance, in Bucha, there is a documented case of like a GSM was disconnected and it was connected through a Starlink terminal which allowed a lot of people to just send like maybe like your 140 characters is enough to tell your loved ones that you are safe. So it's like it's not what I saw is not like providing massive amounts of bandwidth. But in this really emergency situations, it did make a difference, and also in Mariupol, there is also, like, a besieged part of town, the only connectivity left, apparently, is Starlink. So that's ‑‑ in the grand scheme of things in measurement data, you don't see it but it's there and providing probably a very important role.
RUDIGER VOLK: My guess would be that if we see a video link into Azovstal, it cannot be much different than the Elon Musk thing.
CHAIR: Thank you. We still have five minutes for questions. Any other questions? Nobody is in the queue. And three, and two, and one!
.
Thank you very much. Thank you, Emile.
(Applause)
.
CHAIR: It's so good to be back. My name is Jan, and welcome to Berlin. And with pleasure, I would like to introduce the topic that is very close to my heart, and Rene Fichtmuller will tell us how to react to the issues that Emile explained and then we heard what is the situation and now we will hear how to fix it. Thank you.
RENE FICHTMULLER: We try to fix.
First, Emile, thanks for the link and happy birthday!!
.
So, I am Rene. I am working for Flexoptix normally and normally I aware a different colour of shirts, normally this bright orange shirt that Fergus is wearing right now. And yes, how I can explain. Sorry, I am a little bit nervous.
.
We can start how it works here.
.
So, we heard about it since 24th February, the war in Ukraine started, and I was a little bit shocked, it's 900 kilometres away from Berlin to the border to Ukraine, and I have one mindset. Our industry, don't think in colours, don't think in borders in that this thing, we are one family means when I am travelling I am talking to you guys, and ladies, I am talking to my second family and not only to some guys from the community. And that's the reason I had ‑‑ two weeks after the war started, it was the second week of March, I met a friend at Facebook, Patrik, and he was sharing, oh, yes, I want to organise getting goods to Ukraine, I have a truck, also a possibility to have a second truck and he was driving the 40‑tonne truck, I was driving a 7.5‑tonnes truck and, after two days, I think everything was fixed and we started to pack our things into the truck and started our journey.
.
We drove from Berlin via Poland to Ukraine to the border and then to the Ukraine and we drove 11 hours; we had, on the first day two hours of sleep. Then we met, during the night, some people, and we handed everything back‑to‑back trucks, back on back, over into the other truck, and it takes only two hours, and our trucks was empty and we were going straight back with, how I can call it, local work‑arounds to get very fast back to the Polish border, and we drove back to Berlin.
.
And then when I was staying at the customs and we had some challenges between the border from Poland to Ukraine, my buddy was ‑‑ forgot his passport, and I travel two times more than him, and I am a little bit experienced in travels and you need to have your passport with you every time, so ‑‑ and I was going to ‑‑ he started to pass with a normal ID, German ID and it's not enough to go outside of Europe to our countries, you know, and then we got stuck between the borders for a couple of hours and then we prepared the truck for proper ‑‑ then you have seen and it was a second week after the war started, we have seen all these women are coming out of Ukraine with only a suit full of clothes and with babies and the babies was crying. It was direct on the border from Kherson, organising bus trips to Krakow, getting tickets for train to leave Poland and other possible destinations.
.
Then I was back, I think, it was the 13th March I was back. Then immediately I had a call with my colleagues of the Global Law Alliance, normally we are supporting means when you want to organise a NOG, we get everything, everything what you need, like software, ecosystem we are selling on servers, a Cloud system, you are getting Zoom licence during this Covid pandemic, but now we changed a little bit and I shared my experience, and at some point we decided to create a task force and keep Ukraine connected was born.
.
So, the goal of our task force was to check what we can do. It's good to have idea, you can start something but what? And then we started with Facebook and we started with Twitter and Instagram to pose yes, we have a vision, we want to support when you have some equipment you can donate is, ship it to Rene, and ‑‑ but one week later, I had a business trip to Cloud fairs and my wife was calling me and said it's very nice what you are doing, but right now we have a lot of pallets in our front of our house, what we are doing? So she was a little bit unhappy. And also, we had this scenery like Jan, Rene, Sander, idea! Perfect, good, but sometimes it's not a good consolation.
.
After two days, we were completely overloaded with incoming mails, of donations requesting a ‑‑ a lot of people requested support like equipment, money, etc., and then we ‑‑ Sander was, I think, two, three nights very, very active and built up our own supply database and right now, from this point we had this structured way for the inquiries, but not from the real organisation, and then I was happy Jan you, was it, or Sander, was it, had this idea, hey, Nathalie, you are welcome, and I really, really, happy she was joining us immediately and said guys, it's nice what you did, but not in this way. Stop it and we will figure out some organisation details and then right now, everything worked really, really smooth and yes.
.
With the first version ‑‑ this is an example from ask your neighbours for more space. Dropping packets into the van, going to the next door neighbour, deloading your stuff, this was happening every day for, I think, two weeks.
.
So this is our first version of supply and demand database, means all the donors can create an account, all the requesters can create an account, the donors can put all the donated equipment into the database. We have requesters getting accounts, you can create an account by yourself, can claim what they need, and then we have Jan and Sander, they are looking also it fits or it doesn't fit. Also when we have some special requests, we can look what is compared to this what you requested and we can offer it.
.
And then, so this is the Version 2 tool with own registration page, the first one was like you need to ping us via mail. We create this account for you.
.
Also, it's more structured, you can see here what is claimed, what is rejected, what is received, and also we can put in the delivery address, the deployment address, also, all information what we need to make sure this equipment is also going directly to this address who claimed it for the company.
.
Everyone asked me what is your goal? The first goal was collect equipment. And then we had equipment and what are we doing with it?
.
.
So, it was hard work. We worked ‑‑ at first we pinged some people from the Ukraine Internet Association like Nathalia, she is also responsible for the Ukrainian content owner and rights association, and then we found this DEPS crew and, in the end, Nathalia is also working and a good influencer to the Ministry of Digital Transformation in the Ukraine.
.
I want to explain DEPS. DEPS is a local distribution, normally they are working like reselling equipment in the Ukraine and we have a stable and working supply chain in Ukraine. We have own drivers, they have their own couriers, which means we meet this guy behind the border, handing everything over and they are doing everything without any questions. They are getting addresses from us, they are grabbing stuff also during the weekend in private cars, they are going to the cities where the need is and handing over to the local people.
.
Why are we doing it? Or why are we doing this and why the Internet is so important of Ukraine? You have seen it with the presentation of Emile.
Normally, January 2, '21, 85% of Ukraine's population older than 15 years had access so the Internet, means mobile access, fixed access at home, etc.
.
With ‑‑ or Facebook has 10 million users counted on January 2, '21. We have also 7.3 million of Instagram users.
The Internet is a very, very, very, very important also during the war. One of the mediums for communication, it's very important. Like Facebook, Messenger, Twitter, normal e‑mail, chaps, etc. And this is also a picture is not a high resolution picture, but these are a couple of destroyed paths of ‑‑ I'm not sure where it is, but you see all these guys are splicing the fibres in front of tanks, you see destroyed offices.
.
I don't want to be in this situation to splice fibres or whatever, it's crazy and amazing what all the people are doing there. And you can see the splicing in front of a ‑‑ it's crazy ‑‑. This is one of the ‑‑ this was one of the last Mariupol 5G towers, it was a couple of weeks ago. And what happens in April 2022, some guys organised a nice event in Slovenia, call it SSE10, and Jan had this idea, hey, you can come with a truck and you can pick up some stuff on the way to Ljubljana, nice idea, but it's a long ride.
.
So I started a tour on the 14th April. I started on the 8th in Berlin. It was Germany, Czech, Austria, Slovenia, then back to Austria, Czech, Poland, Ukraine, Poland and then back home. It was, all in, 3,100‑3,200 kilometres and real driving time: 44 hours.
.
Then, before I left Berlin, I was picking things up and thanks to the colleagues of BCIX, I was really happy they donated stuff. Also, I saw Matthias from TU Darmstadt., a lot of Cisco gear. Empty house, happy wife, yeah. It's true, trust me. It is.
.
So, then I was going to Ondrej, picked up stuff there, including he arranged a last‑minute interview to Sergey. Then I met Adam is not here ‑‑ I have met Adam from NIXIS at a gas station. We handed over some antennas. And then up next was Slovenia and the most important topic was in Prague, yeah ‑‑ sorry, but in Prague, I met Sergey, and he said, oh, yes, I want to join you to Ljubljana. I said yes, no problem, but only for the record, also for you for the future, never leave the highway.
.
So the truck was nearly full with 2,000 kilogrammes, 2 tonnes of equipment and the truck wasn't really ready for winter. We had 18 degrees in Czech Republic, we expected in Ljubljana 20, 25 degrees, so never leave the highway. It ends up on a high mountain with snow, ice, and all this type of stuff you don't need. Also for the record, when you go up you need to go down on the other side. How long did we need? 3 hours for 15 kilometres.
Then we arrived in Ljubljana. We collected the servers from 6connect, from Jan also, the most needed splicer. It was the first splicer we bought from our donated money, and fully loaded with 14 pallets of equipment and weight, it was over 2 tonnes, I was on the scale, the customs was saying okay, just leave.
.
So, next up. Border Ukraine. And normally you have this ‑‑ I have this ‑‑ I have seen this picture two times, trust me. When you are staying in this queue you need only 7 hours to get around the corner to the weighing scale. And this is 11:32 a.m. And this is 6:20 p.m., I arrived. You can see this is ‑‑ this is the Polish border here, and you can see here is the entrance to the Ukraine border, so no man's land.
And you can see here the border of Ukraine and you also, but this is the second path. You have one path for clearing at the Polish side by customs and clearing, tax clearing etc., those are on the customs side. Obviously the Ukrainian side. So it takes time. And then finally I arrived 11:30 in the morning and I was at the meeting point 7:30 p.m.
.
So we deloaded the stuff like back‑to‑back, I know it exactly, it's very easy and useful this situation. And happy, happy, happy, happy Ukrainian people. Trust me. It's very cool to see in their eyes and this was also one thing I said okay, before we start with volunteers and before we are starting to do more in this, with this project, with this task force, I want to have a level of trust with all the people we are working together.
.
What happens afterwards when I was back? We worked instant, not me, Jan and Sander was working on a lot of things. We added requests directly out of all systems on our website out of the supply demand database, everything is high need, high requested equipment is going to pushed directly to the website.
We have a full Ukrainian language support website also supply and demand database.
.
Multi‑language on the website.
Our daily keep Ukraine connected work?
.
Working on a stable concept for the future.
Before we started this as a task force we had a call initially mentioned guys, we are global NOG alliance, we are not Europe or whatever, we need to make sure everything is also prepared for the future. I'm sure with this project, what we started from the scratch, we can also be an example for the future, also for our disasters. We have experience how we can establish a supply chain or whatever and who we need to talk to, who we need to bring in like local network engineers, local companies like distributors to build up a supply chain in the disaster region. And this is also our goal to fix what you call it, it's like ‑‑ Jan, what was the name? It's like a blueprint to support also other disaster areas.
.
Perfect. This is like a working, a possible work flow, how our system is working with requesting, donations and, when we don't have this compare equipment or the same equipment, what is requested. We can ‑‑ we have good contacts to a lot of vendors, we can request some directly at the vendor stage.
.
Conversation with donors. Right now, we are expecting what is really finished synchronisation is around 15, up to 20 million of donated equipment. We are talking about 30, 35 pallets of switches and routers. Conversations with additional partners, like in Germany we have also like in Netherlands one, I think it's Cloud for Ukraine. I am happy to cooperate with other people, other foundations that are doing the same like us. When we combine our powers, we can be ‑‑ we can have more success like when we are going alone this way, only by ourself.
Conversations with the Polish and Ukraine government and customs. It's very important to have green corridor. Normally we have from the Ukraine in government, we have all this paperwork that makes possible the Polish side can open the doors and let us in the country without any customs or any questions or whatever, but it doesn't happen right now.
.
And normally day work and family. So who is behind the task force?
.
It's ‑‑ a lot of people are thinking I'm doing this by myself. No, I am ‑‑ it's great for myself to work with a great team together.
First of all, Corinne, she is doing all our marketing, communications channel, it's a pleasure, many thanks, Corinne, for your cooperation and for your work.
.
Nathalie, many, many thanks. She is not here, but many many thanks. I am really happy to have you on board in our team.
Sander, well done! Many thanks.
Jan, and at least the driver.
.
(Applause)
.
(Amazing work!!!!!!)
.
Trust me, I am not done, I have nine minutes and a couple of slides.
So, the most important part. For me in person, many, many thanks Natalya from APPK/Ukraine Internet Association, Bogdana, Dmitry and the whole team of DEPS, amazing what these guys are doing, amazing work, 24 hours, seven days a week, they are delivering our stuff to the needed places and the data centres and ISPs.
Team Ukraine Internet Association, Team Ukraine Ministry for Digital Transformation, all the paperwork they are preparing for us. Julia Troyan, also with paperwork, amazing, amazing.
.
Then also, Marchine and Leon from Poland PLNOG, they are also volunteer drivers for the next steps, and Daniel, and we need to respect the rules of war in Ukraine, we can talk about it right now but I think I am sure it's the next days or next week we can say a little bit more about the situation, what happens the last weeks or I don't know...
.
Also, a big thanks to all our donators like BCIX, Ondrej, DE‑CIX, cz.nic, DE‑CIX for power generators, many thanks. I shared with Thomas a couple of videos showing this. Flexoptix for boxes and for transceivers. Thanks, Marcus, Thomas, Matthias and TU Berlin, Global NOG Alliance for everything. NetArt also for technic, NIX.CZ for the technical equipment, US Internet and ZeroTier, and these are our donators, and who wants to support us with money, splicers, equipment? Drop us a mail. You are welcome.
.
What we need and what are the most requested tool at the moment: Splicers, to make it possible to rebuild the infrastructures, the optical infrastructures in the destroyed regions like Mariupol, Kharkiv, one of our splicers is directly in Kharkiv, it was delivered after two or three days to Kharkiv. Yes, we are looking on a clear way money, money, money to buy splicers, we bought from the last donation round, we bought three, they are also ‑‑ four ‑‑ that's better ‑‑. And we will ship them I think the next time to Ukraine.
.
Thank you. That's it, I am done. Time for a beer!
.
(Applause)
CHAIR: Thank you Rene. This was awesome.
RENE FICHTMULLER: It's a pleasure.
CHAIR: Okay, now we have a queue. Jen.
JEN LINKOVA: Thank you, Rene, first of all, for doing this. I have a very practical question as someone who spent the last four weeks dealing with all this logistic hells of delivering stuff across the border. I am very impressed with your system with this database, because I started crafting something like that, so are you going to open source it, make it available for others?
RENE FICHTMULLER: It is already.
JEN LINKOVA: Sorry. Thank you.
RENE FICHTMULLER: You can use it, it's available via GitHub, I think, am I right?
SANDER STEFFANN: Yes, it's on GitHub.
RENE FICHTMULLER: And free to use.
JAN ZORZ: Thank you. Do we give priority to reality or to the queue? Michael Richardson.
MICHAEL RICHARDSON: Hi. So, interesting this is really cool, so, lots of pictures of switches going places, and I am guessing there is some mix of new, used and maybe semi‑obsolete stuff that was donated and went there. And I guess I am wondering what happens to it in six months when it's probably still there, do they get firmware upgrades? That's what kind of worrying me a little bit about, you know, we give them a whole bunch of equipment and six months later there is a zero day and the equipment ‑‑ the infrastructure gets owned, and that would really concern me if that was the case.
SANDER STEFFANN: I will quickly jump in. I am actually talking to high‑level executives of different vendors to get exactly that for the Ukrainians.
JAN ZORZ: Technically speaking, we have nobody in the queue.
SPEAKER: Blake. Do you think, in parallel, there could be some efforts around peering with the Ukrainian networks that are providing connectivity to the networks that you are setting up or that are being set up behind this ‑ in particular, for example, like not everybody obviously has capacity into Ukraine, but I think a lot of these networks are present particularly in Germany, but maybe they just don't have that much connectivity or maybe they need some help with language assistance or whatever, or just like handling their peer queue or whatever, something like that.
RENE FICHTMULLER: It's... I think we had other priorities the last weeks. And we started this conversation about how it looks like in the future when the war is over and hopefully it will be over fast, with guys from the Ukraine Ministry of Digital and Telecommunications to sit down together when everything is a bit settled down and also the completely destroyed areas are rebuilt and talk about like the IXP infrastructure or the landscape in Ukraine. Right now you have also from the Ukraine Internet Association, one proper IXP and then we have I think 15 different small medium sized IXPs that are doing their own stuff without to interact really near with each other.
And I think it's time, after the war, and I'm sure also the Ukrainians have a lot of other things right now to discuss on the table to work on it and not to talk about possible peering sessions. But I'm sure when we have this context and information and when the need is there, we will directly share with the community, absolutely, this is why we are doing this project, yeah.
And I am happy when someone wants to join like for a different topic, like peering focus, whatever, you are happy, I am happy, you are welcome, trust me, it's less work for us and right now it fills a full day right now with work for this task force.
But yeah, also, we can have a chat later after here, and we can discuss some things. Also with Thomas King we can set up something for the future. I am happy when we arrange with the Ukrainian Internet Association like a workshop, how proper peering looks like for the future. Also with the better path of redundancy, it's also the next topic for the next generation of the local infrastructure there.
JAN ZORZ: Thank you.
SPEAKER: I didn't get the app how it's working, so, sorry. Next time. Constanze, from the Ministry of the Interior, Germany. I really want to thank you from the bottom of my heart, it's extraordinary work, and I followed you online and I say absolutely thank you Jan, Rene, Sander, Corinne, it's so great, and we can't imagine what's happened there, and it was a really touchable presentation for me, and thank you for all this. Thank you.
(Applause)
JAN ZORZ: All right. On that side.
SPEAKER: My name is a Marlin Martes, it's my first RIPE meeting. But I just wanted to say that I was just discussing with some members of the community today about moral ethics, and I think that, you know, when we're working with our companies and working in this industry, I think it's very easy to lose a human aspect of, you know, technology and Internet and networking, and I think that when you present something like this, I think it's not only inspiring, but I think it really brings back the notion that we can do a little bit more to help each other and help the world in terms of global connectivity and even, like, helping with the Ukraine, but there are a lot more other places that also could use help. So I think if anything, this is inspirational and an all to arms to anyone else in the community who wanted to do something in support of any other country to do the same thing in addition to the Ukraine. So thank you so much for that.
RENE FICHTMULLER: No problem. That's really why we published this GitHub, to make it public for everyone who wants to use it right now for other kinds of disasters or whatever. Also when you are running some donations or whatever, you can use this tool and we can deploy this tool, and so again, many many thanks Sander for all your hard work you put into the tool. It's amazing. But thanks for your feedback and I am happy when we also can build up a true blueprint for other kinds of disasters and we can also support other kinds of disasters.
RUDIGER VOLK: Well, thanks, great work and good luck. I would like to take a look at some of the not so technical stuff. You reported that the paper war stuff at the Ukrainian/Polish border was worked out quite easily, which is something that has been reported to have happened for a lot of the supplies under the very specific circumstance.
.
Obviously that's going not to be as easy with other potential targets of help when we move on.
On the other hand, I would like to point out for a positive note, that you didn't report any bureaucratic problems at any of the borders your truck was crossing, which I think is actually noteworthy. Kind of, we all, I guess, have been guilty of complaining about EU regulations and rules and bureaucracy, but sometimes it really works smoothly and is very helpful, and we really should appreciate that and, yes, hopefully, we can extend that.
RENE FICHTMULLER: It needs to have a proper conversation with some ambassadors and it was scheduled but our project was not high level enough to have immediately an appointment with the Polish Ambassador etc. I don't want to go into a political discussion or whatever, it's not the platform for it but we scheduled everything and we are working with the Ukraine people together. The Ukraine inside is very smooth when working with the Ukraine people and government together, but most challenging part is the Polish part. I can say you have a lot of planet ‑‑ you have a plenty of time to meditate, 7 hours of doing meditation. But it doesn't help you to get grumpy or whatever, and we had a RIPE call during the ‑‑ I was entering the Polish border and when you see the guys with the guns you don't want to complain, you know. Truth me.
SANDER STEFFANN: So, speaking at a member of the task force, the biggest issue we're having is, like we're getting lots of help from places but the biggest issue is the logistics: Getting storage in Poland because putting everything in Ukraine right now is risky, even the local people there say we can't maintain proper security, stuff gets stolen from warehouses, so we try to only ship things to Ukraine that can immediately be shipped out to the operators. So we are trying to work with lots of different parties, but, you know, big organisations, everything, take time so a big struggle for us is actually getting enough storage space like for 20, 30 pallets of equipment in Poland.
RENE FICHTMULLER: A little bit more at the moment.
SANDER STEFFANN: Even more than that. So that's one area where we could really use some help. The other one is donations from outside the EU where we have to be very careful because if somebody ships over 10 million dollars worth of equipment from the US and we get the VAT bill, we're not going to survive. We don't have the funds for that. So if people have experience with logistics, storage, import law, stuff like that, please get in touch because that's ‑‑ we also want to make sure we have longer term storage because now the stuff that's shipped to the Ukraine is for the short‑term needs but their problems with not going to be over even if the war would stop today, they will have to rebuild for years, so we try to also store stuff that will useful in the future, so if anybody could help us with that part of the work, that would be really welcome.
JAN ZORZ: Thank you. And I see no people running to the mics. Rene, thank you very much.
(Applause)
.
CHAIR: With that I would like to remind you that to support the work of the PC you can do multiple things: A) please do rate the talks so that we know how you like them and what we may change in the future. And until tomorrow, you can put in your name and volunteer for the PC elections that will happen during the week, so if you want to be part of the people who create this night Plenary sessions, or if you feel we are not doing a particularly good job and you want to help us, you are cordially invited. Please send your nomination to pc [at] ripe [dot] net.
And I'll see you in half an hour.
.
(Coffee break)
LIVE CAPTIONING BY
MARY McKEON, RMR, CRR, CBC
DUBLIN, IRELAND.
