Closing plenary
Friday, 20 May 2022
At 11 a.m.:
FRANZISKA LICHTBLAU: Good morning, everyone. Please go to your seats so that we can get started. We know you are all tired, grab a coffee, sit down.
JAN ZORZ: Time for chat will be later. Hans Petter and Daniel. Welcome to the closing plenary. I am Jan and Franziska will take care of this last session, it's Friday. And I have an announcement to do. So we had a PC elections, and the lucky winners are Maximilian and Wolfgang. With that...
FRANZISKA LICHTBLAU: Our first speaker today is Christer from Netnod.
CHRISTER WEINIGEL: So let's get started. I work as a freelance contractor for Netnod. I am a software developer who dabbles in hardware, so I'm ‑‑ I am no FPGA developer but know enough to be dangerous. For the FPGA work I am going to be talk about, I haven't done the development, another company in Sweden did but I know how to talk to those people and take their stuff and build a system out of it.
So, if you know about Netnod, we do internet exchange and DNS but we also have a third leg which is time and frequency and that's the group I am part of. We have six different nodes spread out over Sweden with atomic clocks and redundant time distribution within the site, we have sort of two NTP servers at each site and bunch of measuring equipment.
The clicker doesn't work any more. At each site we have a GNSS for time synchronisation so syncronise our time to UTC SP which is over GNSS and keep it within 20 nanoseconds of UTC SP. We provide NTP and NTS as free services over the Internet and NTP is sort of accurate about milliseconds for most users on the Internet. If you close to an NTP server you can get better time than that but it's hard to promise more than plus or ten milliseconds. We also do PTP for paying customers if they need more time and guarantees of level of service and things like that. But I won't talk more about PTP because it's a different thing.
What is NTP then? It is basically the protocol for time distribution over the public Internet, it's been around since 1985, that was the first RFC and there were proto versions of it that are older than that. It was made in different time so it doesn't really have any security. It does have authentication where a server can hash the data with a secret and that way provide authentication that it's come from the correct server but this requires a shared secret between the server and each client so it doesn't really scale to Internet scales. You can't scale to a billion users. There are technical issues so you can't have more than about 65,000 keys if I remember correctly. There are other security attempts with NTP but none of them really good enough. There is auto key which is getting old by now, it doesn't really ‑‑ it doesn't have good enough encryption for today.
So, what is NTS? It is basically NTP with security. You can make a comparison to HTTPS which adds security and encryption, we shouldn't use HTTP any more and the basically the same is true for NTP.
Netnod has been involved in the development of the NTS protocol so Netnod got involved in 2018, I started working at Netnod in 2019, attended the IETF meetings and helped out with the sort of early proof of concept implementations, and NTS was accepted as RFC8915 so it's proper one now.
What is FPGA? I don't know, are you familiar with FPGAs? A few hands, okay. But it's programmable hardware. It's not quite as fast as doing your own and designing your own silicon but it's often good enough for taking algorithm that doesn't work that well in software and making it almost hardware. An FPGA is basic bunch of lookup tables, a lot of time. With a lookup table you can implement logic and you can basically build anything, registers in soft CPU and you have a lot of these in parallel so you can build register files and all of that. So ‑‑ the next thing to design your own hardware.
Now, on to NTP. You might know this protocol. It's plain old NTP, no security, very simple. It's a 48 byte UDP packet, it's usually comes in Ethernet frame, IP frame, all the normal check sums at that level. All the fields are at fixed positions, there is a mode from client to server, leap second, estimated precision of the server. My name is 100 nanoseconds of UTC.
There is authenticated NTP which adds an MD 5 and/or SHA 1 hash at the end so it hashes a secret together with all the contents so you can see ‑‑ if the server and client has the same secret, they can see, okay, we can trust this party.
Implementing NTP in an FPGA is actually fairly simple. You basically verify all the check sums at the IP level, fill in all the metadata about accuracy, leap seconds and then you can cheat a bit, swap the source and Mac addresses, swap the source and destination IP addresses, fill in the data and just send the packet out. This is actually cheating. We should really send it to the proper gateway but this works well enough in the same routing scenarios.
And Netnod has been doing this since 2016 so Netnod ran public NTPs servers even before 2016 but in implementation of NTP, it can handle 48 gigabits of NTP traffic so has four plus ports and can do wireless port 10 gigabit on each port and this is implemented on a V C709 reference board with about 690 thousand logic sales, which is a lookup table. This reference board is put inside a PC and we are using a PC for control and management and for powering the FPGA.
So this is a picture of the FPGA architecture. So in the upper left corner we have PCI block and that's how we control the FPGA. The PC uses the send commands down and we can read out status on packet counters and all that. In the lower left corner, there is a two NTP clocks, from our we get 10 megahertz and those we feed into the FPGA, and then there's a clock selector so we have a timescale A and B, if something happens with timescale A we can say we are going to use B instead and vice versa. So at all levels we have redundancy in fail‑over. In the middle we have the NTP engines so there is SFP plus and converts the data so the FPGA can work on parallel data at the lower clock rate. On the right‑hand side we have the same thing in process, we serialise the data, take the data from the engine, convert it and push it out through the SFP.
The NTP engine is small and dumb and there are some larger blocks that do authenticated so we support that at wire speed also.
So key points for this FPGA. It's a streaming architecture so everything runs sort of at wire speed. So four different paths, each path does 10 gigabits. There is no buffering, no reordering the latency is extremely predictable, it's the jitter from when a packet comes into the FPGA until the response is transmitted. It's less than ten nanoseconds so if it takes 1,000 clock cycles, that's what it's going to take every time. And the MD 5 and SHA‑1 implementations are huge. They have to be able to do this at streaming speed so they ended up being very, very large because they have to do a lot of things in parallel. So 90% of the FPGA usage is being authentication and the hash algorithms.
What's bad? Nothing really. It works perfectly fine and has been working for over six years. I don't think we have had noticeable downtime on our NTP servers. The only bad thing it doesn't support any security beyond authentication so there is no scaleable security that would scale up to Internet sizes.
So now for NTS, I am going to start with the protocol itself. So it adds authentication and encryption, this is done in two stages, the first stage is NTS‑KE, key establishment, so it's using normal TLS infrastructure, we are using normal HTTPS certificates so you can use let's encrypt to issue a certificate. It's handled by a PC, in our implementation. So and basically what it does is to hand out, they agree on symmetric keys which are then used for NTS time stamping and that is plain old NTP with some extensions for the authentication. It's also stateless which is really important to be able to scale up. And the thing that ties the key establishment and the time stamping together is something called NTS cookie.
So this is how a cookie looks. So there is the server has a secret, it uses to encrypt its data and the server can have multiple keys so it will also have a server key ID that we use. So in a cookie or back to the TLS session, from TLS you can actually make both ends agree on symmetric keys so we have a server declined key C2S ‑‑ what the server does it takes ‑‑ or the client will know these keys, the server also knows these keys, so the server takes those keys and puts them into a cookie and that's the grey part so the server then encrypts this with the server key so now nobody else except for the server can see the contents of the cookie. This is then signed with server key and there's also makes each cookie unique, two cookies issue with the same set of client keys, they will be different unique so you can't really see if two cookies are related.
And here is just a graph of the key exchange, so keys on each side, the client stores it on their hard drive, the server puts the data in a cookie, gives it to the client and it's the client that stores the session keys or the keys related to certain clients, they are stored on the client and not server, this means we don't have to have any storage on the server and all this done is done over TLS Version 1.3.
We have the time stamping request, which is similar to NTP. The first part on the left there is actually NTP request, exactly as in normal NTP. There is a unique identifier for each packet when a client sends off a request it can see the response which also has the same unique identifier, it can sort of figure out which two belong together. The client takes one of the cookies it has received from server, gives it to the server, and it adds its own signature with C2S key.
So back to the cookie. So the server takes the server key ID, finds its server key, used the server key to decrypt the cookie, can now use the client to server key to verify, okay this request I just got, it came from a client that I talked to before. And it can verify the signature and all that.
Here is the response. Same thing here. The NTP response is exactly the same, we have done the same kind of transformation, putting in the metadata, the time stamps. The same unique identifier we had before is put into this packet the server creates a new cookie for the client or it can be told to create more cookies. Encrypts this with S2C key and uses that to sign the packet and this is what gives the authentication so that the client can trust the ‑‑ it can trust the ‑‑ that a response is coming from the server, that it has a relationship with. And this picture is kind of complex and don't worry, I won't quiz you on this, it shows all the steps in the time stamping so up on right‑hand side it's the server storage of the server keys, the client actually has a hard drive which stores the cookies and all the storage is done on the client, so that's the important thing in scalability. And of course on the left‑hand side we have time, which is the goal of this whole procedure, that we want to know the client needs time and it needs time which is authenticated and secure so that nobody has medelled with the time on the way.
So, Netnod has been running NTS servers since 2019 on PC so we basically said we are going to put up an NTS service and keep this running forever, as long as anybody needs it. And we did it on the pre‑RFC versions of the protocol, we have it running on RFC versions now, but of course since Netnod has done all this ‑‑ on we wanted to do NTS in an FPGA but as you saw on the previous slide, NTS is slightly more complex than NTP. One of the issues with NTS is that we are using encryption mode of AS called SIV, synthetic initialisation vector and it's an AE AD algorithm so it is sort of authenticated encryption, so it does encryption and signing and can add additional data to that but it requires multiple paths of the data it's kind of hard to do a fast and good streaming, so basically we need to do buffering somewhere. And that doesn't really fit that well with streaming. So, what we have decided to do was to change the solution and instead of having one large engine running at wire speed we decided to build a bunch of small engines with each one doing something fairly slowly but we have a lot of them and we are having a lot of them, we can get to higher speeds.
So this is the architecture for the NTS, it's quite similar, the same PCI express block for control and status up in left‑hand side corner, the same clock blocks in the lower left corner, we have the same ‑‑ to get data in, same on the right‑hand side to get data out, and then we have all the engines in the middle, and we have a dispatcher and that takes data at wire speed and finds a free engine and hands packet over to free engine to do its processing. On the other side on right‑hand side we have the extractor which checks all engines and whenever one is done with the processing it takes that packet, converts it to the streaming at 10 gigabits and transmits it. So it's a way ‑‑ advantage of this is we can scale, so we can just add another 10% of performance if we can add another engine, with a streaming architecture it's fast enough or isn't, if we wouldn't be able to keep up this would be hard to make this work with streaming architecture.
And that's basically what happens. So some key points for the new NTS FPGA, we developed this on the V C709 on the same board for the production NTP servers. We could fit 16 NTS engines but could only handle about 3 gigabits of traffic and that wasn't good enough; we wanted 10 at least. So we did a simple thing and threw money at the problem and bought larger, he got an XI LINX which has about 2.4 million logic ‑‑ so it's about three‑and‑a‑half times larger, three‑and‑a‑half times so we could fit 40 NTS engines and do 10 gigabits of traffic. This production this runs on Arista switch, that is a product from Arista which is the same FPGA on the reference board but in a rather nice 1 U rack unity with redundant power supplies and all that, so it was quite a relief for us not to have to build our own PCs and build the thing itself, so it's a bit more expensive but we buy convenience by buying a Finnish product.
And here is the sort of only coloured picture you will see in this presentation. This is a view of the physical layout of the logic cells on the FPGA so this is a view produced by the the development tool we are using for the FPGA. Each colour block up here is one of the NTS engines so there are supposed to be 40 of them, I haven't actually counted, but I hope it's true, so each engine is sort of ‑‑ all the logic for one sits very close together and 40 engines are sort of spread out all over the chip. There are some small parts here where you can see the 10 gig 5 which is a small tiny light blue blob. The PCI block is larger because that is a rather complex protocol. The dispatcher, you can't see it because it gets laid out together with each engine so they are small blue specks in all specks in all the engines but it's hard to see. Black spots in it, that's the extractor which collects all the data and feeds it out of the 10 gig, and don't quote me on this since I'm not a FPGA expert but there is a lot of black in there I think that means we are probably not using the resources as efficiently as we can here, so we might be running out of routing resources while we are not using all the logic, we might be able to improve this a bit.
So one thing regarding NTS, since we have gone from a streaming architecture with very predictable latency we are now doing lots of small engines, data can be ‑‑ packets can be reordered so the jitter can actually be up to couple of micro seconds. In practice this doesn't really matter because the first router in front of the NTP server is going to add micro seconds to jitter. If you are a few hops away on the Internet there is going to be a lot of jitter because packet reordering are still getting behind others and the same thing is true for NTP so for bragging rights it would be nice if we could have kept the jitter down on NTS but well, it doesn't matter in practice.
So for sort of looking at this, we could maybe do some improvements. We have quite a few engines, 40, so the dispatcher and extractor, they end up being rather large and complex because they have to take data from a lot of places at wire speed 10 gigabit and they end up being rather big and as I said, maybe not that efficient for the extractor. Maybe we could take the engines and make each slightly faster. Speed it up, so if we can get the number of engines down to 20 instead of 40 we could make the dispatcher and the extractor smaller. On the right‑hand, it works right now and fulfils our requirements so I am not sure we need to do that.
So, for some kind of summary about this, well the FPGA, NTS FPGA works quite well, it can do 10 gigabit on Arista switch, both NTP second Chrony have support for NTS. Be aware the protocol has changed from the draft to the RFCso the draft version of NTS are not compatible with the RFCversion so you need a new enough version second Chrony. If you run Debian 11 bull's‑eye they are new enough. I think should have new enough versions. I believe Fedora has new enough versions.
There is supposed to be support for NTS in system D, in time D, I haven't tried it out myself but I did some code review of it when they added support so hopefully it does work.
And when it comes to our implementation, it works quite well, there is room for improvement and me being sort of an engineer I always sort of would be nice to make it perfect but you can't really. So, it does work and we are quite happy with it.
So, final slide. As I said, I am with Netnod in the time frequency group, links to that, links to some NTS white papers if you want to read more, all the source code we have done for the FPGA and the servers are, it's Open Source, we are using BSD licence and all available on GitHub so go there and take a look. We have our production NTS servers running, 1 and 2, we are planning to roll it out to our six sites in the future, there is a bit of supply issue with hardware right now, as you know. And finally, if you don't come up ‑‑ if you come up with a question later on feel free to mail me. Thank you.
(Applause)
JAN ZORZ: Read first from ‑‑
FRANZISKA LICHTBLAU: I can start with written questions. Two questions by Kurt Kaiser in his private capacity: This FPGA is a kind of converter from NTP with NTS with accurate input lines but has no logical hold over option, is that correct?
CHRISTER WEINIGEL: It's an FPGA that implements NTS protocol. When it comes to hold over time we have clocks providing the 10 megahertz and to the NTP and FPGA, so basically we have what is it, PR TC 100 second nanosecond hold over, I think we can do 90 days, it's done by the clocks and not by the FPGA itself, it takes a clock input and provides time stamp packets with what we get from the clocks.
FRANZISKA LICHTBLAU: The second question from Kurt is: Did you also consider implementing NTS and P 4 fabrics, I would imagine it would be a good fit for that?
CHRISTER WEINIGEL: I am not a network guy and I don't know what that is, and I didn't see that presentation.
JAN ZORZ: So please, Tom.
TOM HILL: It was a slightly different one, from British Telecom. I was curious I think mostly how you settled on starting this FPGAs given the relatively high barrier to entry for getting into coding on them and I thought personally about using networking, slaving NIX ‑‑ things like slab switch which are really, really extensible. Was there a situation where you said we can only do this with FPGAs? Is it the specificity of the latency?
CHRISTER WEINIGEL: I can't really answer your question because Netnod started using before my time, I think back in 2014, 2015, before the ‑‑ and I started at Netnod in 2019. One reason I think is that we wanted to basically be able to survive denial of service attack. We said that if somebody does a distributed denial of service attack against us we should have enough bandwidth ‑‑ traffic available so we can actually survive and at least respond to some of the people asking you. If you ask us ten times it's supposed ‑‑ supposed to get through at least once. So I think it was the idea of make our ‑‑ make our hardware so fast that it's impossible to bring it down.
TOM HILL: Today I learned that Arista sells switches with FPTAs in which seems quite ‑‑
FRANZISKA LICHTBLAU: Another written question: Will the FPGA solution also handle NTP Sec over IPv6?
CHRISTER WEINIGEL: No, we only do NTS over normal, IPv4, IPv6 today.
SPEAKER: From the global ELI foundation. How much latency does this whole encryption story as to NTS in comparison with NTP?
CHRISTER WEINIGEL: The latency I don't know, actually, it's not that important because in NTP you have received time stamp and transmit time stamp so any time it takes for the latency, we can just subtract it from the time estimate so it doesn't matter. It's jitter, how good those are, that's what matters. It's a couple of micro seconds instead of nanoseconds, as I said it doesn't matter in practice because the first hop will add more jitter.
Gordon: I am fairly a layman when it comes to hardware stuff like FPGA so I'm just wondering, when it comes to FPGA actual implementation, do you actually take into account the locations of where the FPGA compiler puts the various engines within the package itself? Does it really matter and do you actually get any gains from it if you doing this?
CHRISTER WEINIGEL: Yes and yes. Physical layout is very important because the ‑‑ light travels at about 200 kilometres per second in copper so basically this much is one nanosecond, so if you have lots of hop long routing it's going to take longer so you have to consider physical layout and FPGA optimisation is really, really hard, and it is possible to do guider optimisation where you tell the tool where to lay things out and I'm not good enough to do that and if you have to do that you have a tight fit and it's going to be hard so usually you want to try and stay at 80, 90% utilisation because the tools can do it automatically because it gets really painful if you don't.
SPEAKER: I see, thank you for satisfying my curiosity and thanks for this interesting talk.
CHRISTER WEINIGEL: I will be here if you have any questions after this.
JAN ZORZ: Thank you very much.
(Applause)
FRANZISKA LICHTBLAU: Next up is the tech team reporting on this lovely meeting from their perspective.
SJOERD OOSTDIJCK: Hello, everybody. My name is Sjoerd Oostdijck, I am the technical coordinator for this meeting. And if you don't know who I am, then apparently we did a good job. I have gathered a couple of interesting statistics for you guys. We got gigabit uplinks from Deutsche Telekom and while I was sitting at the help desk the only question I got or one of the first questions I got from them is is something wrong, we don't see that much traffic? So we requested, as far as I know, a gig link but they gave us 10, and looking at the stats we are only doing about 120 Mbit on average during conference hours so it's not really that much. There was a nice spike at some point, about 721 Mbit, I am not quite sure what it was, maybe Apple were rolling out an update or something, but most of it was actually over v6 so that's pretty interesting.
V6 has been doing very well this meeting anyway but we will get to that. Here is a little drawing of what we put together this week. We have basically a switch at the core of it and take your time if you want to look at this, you have questions, come find me or e‑mail us, my e‑mail address will be at the end.
We rolled out 45 access points, I have not heard anybody complain about the wi‑fi so I hope you guys all had a nice experience.
I think they are getting nice and warm so if you ‑‑ if you are feeling irradiated, that would be it.
So like I said, IPv6 has been doing very well. If you look at the graph, this is one day of traffic. But actually the green bits are IPv6 and the red bits are IPv4. And the green bits are actually winning, so I think that's pretty good news.
So Jen also is enthusiastic about IPv6 and she found me, I don't know if she is still here, she found me yesterday all excited that the wi‑fi at the dinner was actually v6 only, we checked, it's true and somebody did it but we have no clue who. There is is the ‑‑ okay. So that's pretty nice. We also have a new webcast set‑up, I made a professional diagram. So most of it is actually the same but it's Meetecho that you all know and love from the, you know, meetings during Covid, right? And actually, it's basically the same thing over here because the screen we are looking at now is just a raspberry pie, you know, watching the stream. We did bump the stream a little bit because the quality wasn't that great so it's the slides are 2 Mbit stream and the talking heads are much lower but I have no heard any complaints. Apparently everybody is able to watch 2 Mbits. If you can't you also have a hard time using Netflix and those guys.
Some stats about the webcast, how many people. 45% over IPv6 is not quite over half but it's also not bad at all. And only 44% Macs, I think it's going down. Coffee statistics, also very important.
(Applause)
I was told the record was somebody drinking ten espressos a day, so anybody more? No. Not willing to come forward, okay.
And of course I would like to thank the tech team, I think we all did a smashing job together, you know, if you have any complaints then please hold your tongue.
And of course the web team, the Meetecho team and the steno ladies as usual.
(Applause)
That's it. Any questions?
FRANZISKA LICHTBLAU: We will start with an online question. There is again Kurt Kaiser: Why just two MPS for video when there is two times gig sitting around, cost for Meetecho distribution I would assume?
SJOERD OOSTDIJCK: We want to keep the stream reasonable for people that are out there, and maybe have a bad connection and don't have too much bandwidth. Not all countries have super duper Internet, and want to be inclusive, even if you have bad Internet you can watch. I did hear some talk about maybe having two flavours of streams like a high quality one and low quality one for next time, so hopefully that will happen.
[WILL]: IS 26 ‑‑ I was wondering if you had any way to protect yourself in case of a DDoS attack on those two 10 gigs because I know some user that have 25 at home and they could fill the pipe.
SJOERD OOSTDIJCK: No, I think we just have to watch or ask Deutsche Telekom or DE‑CIX or somebody else for help, but you can win, it's, at the end of the day, it's just a couple of virtual machines routing and ‑‑
[WILL]: Because if we are relying on this platform to have like the remote participation and so on, it will actually split the community if we have got like some issue, and I am not talking about the nice people here in this room, other nice people around there in the Internet.
SJOERD OOSTDIJCK: I think the easy solution is we upload the stream to some servers in the Cloud and distribute everything from there, so if somebody were to flatten our connection it probably just hook up a 5G mobile phone and stream it like that.
JAN ZORZ: We have another online question and then it's you.
FRANZISKA LICHTBLAU: I think that is more a general question for everyone because Elvis is asking: What happened to spatial chat, nobody in for whole week.
JAN ZORZ: Everybody is here apparently.
SJOERD OOSTDIJCK: Nobody was there so I didn't go in to look.
SPEAKER: Maximilian, speaking just as a participant. What about wi‑fi at the top floor, could you have managed to do that or was there some technical reason why that didn't work, it was pretty unfortunate up there at this one social event in the top floor where the mobile data wasn't working properly, the hotel wi‑fi has password protection and our wi‑fi wasn't here. Any thoughts?
SJOERD OOSTDIJCK: So normally, well, it was the Tuesday party, normally we don't bring wi‑fi to the events and in this case actually I think Covid got the better of our initial venue, so the top floor was actually the fallback and we just didn't take it into account that we could have brought an access point. Yeah, fair point. We will think about it for next time, perhaps, if it's in the same venue, because, unless we can get sponsors for bringing 10 gig to the dinner or something, I don't know.
GERT DOERING: I have been to many, many, many RIPE meetings and I think this was the most flawless wi‑fi I have ever experienced so it just worked, it was so transparent perfect that it worked.
SJOERD OOSTDIJCK: Coming from you, that's a compliment, so thanks.
DANIEL KARRENBERG: Just myself. Just not to have only compliments. Can you explain what the purposes of adding this random slide to the slide clicker?
SJOERD OOSTDIJCK: That's because it's actually the Meetecho stream, so you click it here, it goes to the Mac over there and then, you know, that thing advances the slide, but the stream is just a couple of seconds behind so that's the delay.
DANIEL KARRENBERG: Maybe instruct the speakers a little better next team.
ANNA WILSON: I want to say thank you particularly for the way the videos of the talks are posted so quickly because it makes such, it's such a cool thing to be able to watch a talk go that was amazing I need to share it and by the time I hit the website it's already there, thank you so much.
(Applause)
JAN ZORZ: All right. I see nobody else running to the mics.
SJOERD OOSTDIJCK: I have one more quick remark. The truck to come pick up all our stuff is quite early this time so if you want to print a boarding pass, I don't know, do so before lunch. Because we are going to have to take the network down quite quickly. Is anybody going to print a boarding pass? I don't know.
FRANZISKA LICHTBLAU: Are we still doing this?
SPEAKER: The airport is actually open.
SJOERD OOSTDIJCK: With the storm you mean.
JAN ZORZ: Rudiger, Secret Working Group didn't start yet.
(Applause)
FRANZISKA LICHTBLAU: With that, the Programme Committee hands over responsibility of this session to our lovely Chairman, Mirjam, and we would like you to remember you can still rate the talks, we love your input, and with that, over to our Chair.
JAN ZORZ: Thank you very much.
MIRJAM KUHNE: I am new to this, I still need notes. Thank you, thanks for running this. I think this was a fantastic programme, maybe first of all thanks to the PC and for chairing, the session Chairs also this morning. It was good to have some good content this morning so people got up and were in the room and listened to the presenters, which was great.
This is coming to an end, it's great to still see so many of you, and I am just going to walk you through some more statistics. We had totally checked in attendees 769, which I don't know compared to previous physical meetings, but it looks pretty high, especially we also 180 online attendees which was great we had 211 viewers per day and 228 newcomers which is a good number, 179 on site and 46 online. We had only one child on site but that's also because we were a little careful still with bringing kids to the event with Covid but we had eight online children for the childcare which we carried over from the remote meetings so that's still being used.
I included here a report from the trusted contacts so we had the four trusted contacts here during the week, and they were, they gave me some numbers, I am going to look at my notes here. So there were two direct approaches, one in person and one via e‑mail, the e‑mail complaint was resolved to the satisfaction of everybody involved. The other complaint is still waiting for resolution. And there were other several reports from third parties brought forward to the trusted contacts. Most of them were related about ‑‑ they were related to aggressive tone of voice used at the mic phones. And then there were some other reports but they were the main point. And I think, I mean, the fact that we have more reports than, it seems like more reports than we had in the past, because they are reporting about it better, I think it's actually a positive sign, it looks like people are looking out for each other and noticing things and going to report them and also I had the feelings the trusted contacts were approachable and visible and we had this posters out there so they were easier to find.
Right. Some more numbers. So these were the participation or the country distributions from on site participants. Most of them were Germany, not surprising. 10% of people from the Netherlands, that's excluding RIPE NCC staff, so regular participants, what I find interesting, 37% from other countries, which is actually quite a diverse group of people, I think, so it's not the usual suspects, which is nice.
And for online it's even more diverse, 62% of all online participants were from other countries than the top five that you see here on the list but Ukraine is also part of the top 5 ‑‑ 6, 5, whatever ‑‑ 5.
So that's good sign.
I wanted to bring up, because this is the closing plenary and this is what we usually do in our new task forces and Working Groups, Chairs stepping down and this one is about the diversity task force and people have been wondering about the status of the diversity task force and I want to give a quick update here. The diversity task force was set up in ‑‑ at RIPE 74 in Budapest and they came up quite a long list of action items at the time and many of those have been implemented in the meantime, not least the Code of Conduct which was set up as a separate task force, so we have a new and improved Code of Conduct in place, and the task force is still continuing to work on the reporting mechanism.
Other things that were implemented to increase and help with diversity is the childcare, mentorship programme, the Women in Tech sessions and so we have recorded all this also on the task force web page. And thanks to everyone who has contributed to this in the meantime since RIPE 74.
Now, at this point I think this work has evolved into a more ongoing community activity rather than the work of a particular task force, which was like 10 members or so so it is something that the community at large needs to continue and pick up so therefore I propose here to actually dissolve the task force as it stands but maintain the mailing list which has been there all along, it's more than these ten people on there, there are a few hundred on the list, diversity [at] ripe [dot] net, it's an open list and everybody can participate and so I am proposing to keep that list and use that as a basis for continuous work to increase diversity and improve inclusion in the RIPE meetings. Everybody is welcome and I am looking forward to new and constructive ideas on the list and then we can see, based on that, how we ‑‑ if you can maybe have regular reports to the community or from time to time a meeting, maybe the task force ‑‑ the list, the group, the community, can also take on the responsibility to organise these diversity in tech sessions, maybe we don't want to call it Women in Tech, there was one feedback we received. So there's a lot more work to be done and I would kind of like to give that back to the community. Are there any ‑‑
(Applause)
That was my question. Do you agree with that? I guess you do, unless there are any questions or objections we will officially close the task force and mark that on the website. List the successes and open action items so that can be picked up.
There are two Chairs, Working Group Chairs that have stepped down during this meeting, it's Constanze Dietrich, but thank you very much to run the task ‑‑ the Working Group together with Sander over the last few years and have some ‑‑ trying to get continuous activity on the Working Group, and the other Chair stepping down is Brian Trammell from the MAT measurements automation analysis and tools, MAT, whatever. So they are stepping down. It's called groups were called former such‑and‑such rather than creating a new name. Anyway thanks Brian and Constanze, I hope to see you back here in the community.
(Applause)
Of course we also have some incoming Chairs, Massimo Candela and Stephen Strowes, he wasn't here this week, he was participating online but Massimo is here and welcome, those two. And Peter Steinhauser for IoT stepping in for Constanze, we have Markus Du Brun who is the third Chair for the Anti‑Abuse Working Group and had the pleasant task to chair this on his own because the other two couldn't be here. So welcome.
(Applause)
Here you see the full list of Working Group Chairs that we currently have and you can see on each of the pages when they started and what the terms are and when the next round of nominations will take place and the biography of the Chairs you can see there.
Also many thanks again to the PC, this is the current PC that put together the agenda for RIPE 84, and I was told that there's some gifts for you. Where are three? Those of you who are here, Franziska, I can see you and Jan and Peter and Wolfgang earlier, I see Dmitry there, Alex has some surprise gifts for you. Thank you very much for the work.
(Applause)
That was particularly stressful this time because there are only a few PC members here at the meeting and I saw them struggling and I was shifting around presentations, some presenters couldn't make it and they had to be very creative to put together the programme.
Unfortunately Peter Hessler and Wolfgang Tremmel are going to step down, but ‑‑ welcome again Wolfgang to the PC, lovely to have you back and new member on the PC is Maximilliano, some of you might know from other types of works he is doing. I don't know if he is here. Also thanks for all the other nominees, we had quite a large group of nominations this time which we were very happy with because last few years we were struggling and it's great to see so many of you are interested to contribute to the programme of the RIPE meeting, I think that's really essential.
Now, more gifts and prizes. So we usually hand out prizes at this point, I am going to read out the names, you don't have to come up here, the meeting team will contact you and find you and send you your prizes.
Korina, I haven't met you in person, first online registration...
From those who rated the presentations, we have the winners...
Applause
And also Kahoot winners are listed here. We couldn't find the person behind A ‑‑ so if you are around and you know who you are, contact the registration desk and they will hand you over your gift. Patrik Scheck won two, so congratulations
(Applause)
There is still time to give us feedback about the meeting or aspects of it, not just rating the talks but also the meeting in general, what you liked and you didn't like. I have to admit I heard a lot of positive feedback this time and people were generally happy to be here and see each other and maybe that's reflected in the relatively low traffic that Sjoerd was presenting, people were talking to each other in real life. And of course many thanks to our local hosts and DE‑CIX and all the other sponsors that you can see here on the slides, who helped us to fund various parts of the meeting, the lunches and the coffee breaks, connectivity and everything else that's needed for this meeting, so thank you.
A few other groups of people we couldn't live without, that's the registration desk, here are picture of them, you have probably all seen them outside. Big applause for them.
(Applause)
Stenographers.
(Applause)
And the people behind Meetecho, we had two of them here at the meeting.
(Applause)
And some more behind in the background. And of course also big thanks to our tech team.
(Applause)
I think they managed to do this last night without big injuries. No? Well at least the tech team did.
Right, so I think this is almost ‑‑ this brings us to an end, and last but not least, I hope to see you all back at the next RIPE meeting in Belgrade in Serbia where we will be in October.
(Applause)
So we haven't been in that region for the RIPE meeting in quite a while and we have the southeast European meetings there of course but it's great to be back there for RIPE meeting so I am really looking forward to it and we have a bit of a surprise, not you, no, we have actually a bit of a teaser from one of the hosts of the RIPE 85 meeting, if you are here, then I would like to, I would like to give you a bit of a teaser to encourage you all to come to Belgrade.
(Applause)
SPEAKER: Hello, everybody, I am Jack ... from Serbian registry and I will be local partner together with Serbian exchange, internet exchange, to help RIPE to set up this meeting. And I don't know how much ‑‑ how many of you were in Belgrade, Belgrade is capital of Serbia, south part ‑‑ southeast part of Europe, and this pretty ‑‑ pretty fun place to be. Yeah, I know. I have to tell you because mostly I was getting that question from Europeans, but from people oversees, they ask are you carrying guns? No, we are not carrying guns and we are not shooting each other on the streets. And we are usually really good hosts.
Those are some facts that even I did not know about Belgrade. Belgrade is one of the oldest cities in the world. The first establishments were there more than 7,000 years ago, and it is the oldest continuously inhabited city in Europe. It is the largest city of Serbia and has around 2 million people. You will see some pictures because if I tease you, I have to tease you with nice pictures and nice area, and the city doesn't have that ‑‑ actually, it does have two different parts, the old city, with some buildings from the end of 19th century and most of them are after the Second World War and new Belgrade, which is social city and we call that sleeping area for Belgrade but it is developing now and business centres so it looks much better now than some 10 or 15 years ago. Belgrade is located at the crossroads of the roads from east to west and from north to south. At the confluence of two big river venue, biggest European river and Sava which is biggest river.
Imagine that most buildings in all Belgrade are from ‑‑ all date 18 ‑‑ 19th century, but we have to thank a lot of wars that we had there, and for ‑‑ to the history of Belgrade, Belgrade was destroyed 44 times and rebuilt again. So you cannot expect to have old city, but there are some. And there's Belgrade fortress, first built by Romans over there and with some addition maintained by ‑‑ and it is in pretty good shape and it is landmark of the city.
What's interesting, and you will see when you come, we have a lot of restaurants, bars, coffee shops, and they are full all the time and probably we have to say thanks because first restaurant actually, Kafana, was established in Europe was established in Belgrade in 16th century.
What you should expect in Belgrade: First, superb RIPE meeting. How I know that? Usually RIPE meetings are excellent. But we will work with Programme Committee and see what we can do to make this one really superb.
Having good meeting, it is not only good presentation and interesting topics, but that's a good time during and after the meeting, so I mention a lot of restaurants, a lot of coffee shops and there is good food. Serbian food is mainly meat, we have joke, you know ‑ came in a restaurant and they ask waitress, we are vegetarians, what's the best thing to order here? And the answer was: Taxi. Don't worry, there is food for everybody. There are specialised restaurants for vegetarians but in ordinary restaurants you can order vegetarian food, which is really nice. We have all sorts of vegetable and fruit stuff which is really nice and adding meat for meat‑eaters, it is really great. Be careful, portions are large, so ‑‑ and it is not expensive, so order as much as you can eat and don't overeat during the time in Belgrade.
Also, there is this distinct sight‑seeing, different part of Europe, different than Western Europe, I don't see many things which are close to Italy or Spain, but it is really nice part of the world to see. And also there is ‑‑ there are various entertainments during the night, Belgrade is known as entertainment city, you will see pictures, and two big rivers, we have lot of floating restaurants and clubs and there's fun all the night.
I hope that in late October we will have nice weather. It depends from year to year, we cannot know how it will be this year but sometimes it knows to be around 24 or 25 degree at the end of October, without rain, and sometimes it is raining and really cold around ten, average temperatures at that time of the year are between 18 and 20 degrees, I hope, will be there, without some bad weather. And some nice pictures, this is an aerial picture, city, central part which is the biggest church in Serbia and the Balkans and the biggest orthodox church in the world.
This is the river and there is nice promenade on new Belgrade that far side top of the screen, it is new Belgrade and the lower part is the old Belgrade. This is a lot of restaurants and the small picture there, those glasses are special glasses for drinking in Serbia, and you should try that. Pity not a lot of places still hold that but it is fun drinking that from them.
This is picture of delta of the Sava, bottom left, into Danube, but on this picture and ‑‑ it doesn't look that, it in life it looks like Danube is coming to the Sava and not vice versa. In the middle Big War Island, I don't know why it is called that, no big battle, that island is well‑known for being amount of different species of birds and it is protected area for them. Sometimes also some wild animals from island swim to Belgrade and we have deers, not often but sometimes you can see deers in the centre of the city. They are scared, of course. This is one of the entrances to the Belgrade castle. This is the church that I mentioned. Outside it is monumental but from inside it is not painted but there is mosaic, you should go there, also close to the hotel where venue of the RIPE meeting, very close, is museum, you can see it is nothing special but, you know, maybe it is interesting for people to see the museum.
This is picture from the museum and those are local partners run it, which is Serbian domain name registry and it is good example for Serbia, how to build something, it is community‑built organisation and it is driven bottom up, main tasks are domain registration and DNS system for TLD, cybersecurity in regard of domain names and DNS and helping Internet community in Serbia to do better work and to grow. And internet exchange for Serbia and really good engineered organisation, it is, I don't know, some ten, eleven years old, but it is growing every year, helping many organisations to connect and also as ‑‑ want to say connecting people and businesses. SOX has multiple hundred gigs connection to Vienna, maybe in Frankfurt and Sofia and really, really good driven and you will hear much about organisation next RIPE meeting, but for now, that's it. And I hope to see you in Belgrade and we will have good time. If you have any questions...
(Applause)
I guess no questions? Thank you.
