Hallo Du!
Bevor du loslegst den Talk zu transkribieren, sieh dir bitte noch einmal unseren Style Guide an: https://wiki.c3subtitles.de/de:styleguide. Solltest du Fragen haben, dann kannst du uns gerne direkt fragen oder unter https://webirc.hackint.org/#irc://hackint.org/#subtitles erreichen.
Bitte vergiss nicht deinen Fortschritt im Fortschrittsbalken auf der Seite des Talks einzutragen.
Vielen Dank für dein Engagement!

Hey you!
Prior to transcribing, please look at your style guide: https://wiki.c3subtitles.de/en:styleguide. If you have some questions you can either ask us personally or write us at https://webirc.hackint.org/#irc://hackint.org/#subtitles.
Please don't forget to mark your progress in the progress bar at the talk's website.
Thank you very much for your commitment!

================================================
Herald: 
 Our next speaker Sergey Gordeychik. Sergey has been doing security research products and services for the past 15 years, more than 15 years. Since 2011 he is director and script writer at Positive Hack Days Forum, the largest cyber security event in Eastern Europe. Sergei has for instance been working at Kaspersky Lab and Positive Technologies. He's also a visiting professor at Harbour Space university in Barcelona and leader offthe SCADA StrangeLove industrial cybersecurity research team. today Sergei will talk about how to hack software defined networks and keeping your sanity while doing it. Let's give a warm round of applause for  Sergey Gordeychik.
 
Sergey Gordeychik: Hello. Hello. Good night. Let's start to refreshing memories. This a big honour for me to speak on the 35C3 simply because my first talk here was on 29c3 with SCADA StrangeLove team and I think I can skip this introduction thanks to our host because everything is here and what I want to say about me still I am a Russian living in Abu Dhabi and do Overstock(?) because I saw this album in the airplane when I fly here so accept bitcoin only. So let's start to talk about software defined networks. Well the software defined the networks in general and is divine in particular case is a magic. So according to Gartner it will kill MPLS. It will replace all your Cisco and Juniper device or Huawei if you prefer Chinese, but it is bad you know it's a problem last year and it will solve all your network problem because it's have AI inside and it will magically optimize network operations and do everything, including security. So because it's perfectly safe to implement in wide-area networks efficiently and securely. So OK sounds good. What's this actually software defined networks? It's so simple! If you're familiar with routers and switches like it's just accept packets and send it afterwards somewhere according to routing tables or MAC tables. But software defined networks is quite different. A
nd when we trywith our team to understand how it's work our first impression was like this. We are hackers. We don't want to deal with this shit. But the only challenge we met before you hack something you need to actually activate it and make it work which is why the staff understand how software defined networks and as SD-WAN works. So what's the main difference between traditional land and then SD-WAN one? In traditional land you have different device which solve very specific purpose for instance switches or routing or firewalling or network load balancing. In case of SD-WAN you have just a server which runs operating system, in most cases it's like Linux and on the top of this operating system you have very specific models like CPE which do specific network functions, that can be firewalling and it can the routinng, it can be switching, it can be network load balancing. So you replace specific devices with one big server which magically do everything with AI and in the cloud. Sure. 
So in that SD-WAN we have several layers: so low level is data plane when actually you process packets and decide how to go it in which way how to firewall or drop it. We have control plane which manage different routers different devices. We have management plane which can help to apply policies. And orchestration plane because it's serious things that should have something which called orchestration. But on the technical plane again we have hardware with operation system and the layer which called Network Function Virtualization. What's  Network Function Virtualization? The way to apply different network functions sorry to this specific device. It's really useful for instance to the network operators who provide you with the specific box and if you want to activate any functions it can be a web application firewall or the sandboxes we just upload very specific virtual machine, It can be a docker, It can be KVM image to your hardware and you can start to use it 
because inside
 of this box you already have all the system infrastructure which process packets and  pass it from one virtual net or function to the other. This helps to organize things like service chaining so you can distribute different network functions on the branch level, on the cloud level or on the HQ level. For instance things like the content filtering which can be very heavy from performance perspective can be distributed. As example on the branch level you can use simple things like that antivirus to process content, on their HQ level you can use more hairy things like sandboxing and if antivirus or specific rules see that this content is suspicious it's forwarded to HQ through the MPLS or other VPN and next process it in the HQ. Or you can also analyze it in the cloud for the simple things like a cloud threat intelligence but where actually your SD-WAN box will send MD5 hash to the cloud and check is it good content or bad, or sent all files to the cloud to double check. 
Not bad. And I think that is why SD-WAN become more more popular and you can see that even military guys in the US will decide to switch to SD-WAN because of security, cost saving and all this benefits. Okay security sounds very familiar for us and we decide to obviously hack. I think most of you have experience in hacking of different network appliances and you know that sometimes you need to have complex things like an soldering kit or debugger, JTAG or other etc. But not in this case because SD-WAN actually is not appliance, it is virtual appliance and to start hack it all you need is to go to the AWS or Azure and just activate with virtual plans for, I don't know, 10 bucks per month and next step is get root get root on it.


TODO BELOW

 It's where a good talk presented on different conferences including zero nights how to hack virtual appliance and use service select a checklist for our research. Because if you can get all the planes you already have access
 to this syste
m you can mouth file system to the other two on planes you can grep UTC schedule you can find a lot of different back doors just through the static analysis but all the good things start from the Google for instance to find admin password for one of that is the one appliance we just google for hop and found what most of the scripts which use it to automate these are plants use a user name administrator and password so one two three and actually if we found what this user name and password is hardcoded where because there is no way to change it. Next step to a router it is just to Google for old vulnerabilities. For instance in Silver Peak we found what guys had reported vulnerability in September 2015 and it's to work in in March 2018 so Google works because Google is strong with this one. Next thinks is grip its hopeless work as a string you and using very open called the code word password. You can find a lot of interests and think like hardcoded passwords in different location in the configuration files in the database connection string can the system logs because it can. It's took a place and some of had deployed before you start to use it. So in the lots of it a lot of life interests in the formation in the shadow file like in the one of Cisco appliance decision or file which use this encryption in the files and made him. You can do some forensic because again if you get to the appliance. Someone had deployed and sometimes a chain to hide it is and you can see that the cut in the bush history you can find that someone or a ram scrub doubleU.S. shell script which actually set up a different passwordetc. So if you somehow can recover escape you can find a lot of interesting information that these kind of passwords of admin users and you can see it from this password. You can find the hash and next strike a brute force it just was my guess about maybe because others have passwords versus one two three maybe other network appliance like silver pick have similar pas
swords similar pick one two three and this guest was successful and it's good because you cannot stop the progress if you have experience heavy threat teaming in the enterprise network you know about Cisco Cisco or Huawei if the administrator and hallway you want to freeze quite common passwords in this case it's more complex things sometimes where a leak like this tottering network stuff you know. But still if you did not get the root the three simple steps again swift to the plants and you can always patch it so you can change for instance Precious is the is a shadow you can change good script you can change remote management configuration the password and next in this configuration and get through the password to do next step. Security assessment. So security assessment at the beginning we did in very you know most scientific way. We just how close are things. But after all we did. Some let's say scientific research and we have an article I would give you a link like as the one threat landscape. The step by step acceptance assessment what do you X should Huck to get maximum results. But let's mix this thing with funny hacks and the scientific approach. So from the system engineer point of view as Taiwan have hardware which of the hardware operation system in most cases again off the shelf Linux and different digital services. Let's start from the operation system because against again everything you saw in the recent talk about the BMC and the remote measurement interfaces related to hardware it's two works here unless it disabled by the vendor. But it's highly unlikely it on operation system. Now we did the very simple research we just check the patch level of the all components installed on this box and then you can see about budget level that you can sleep. Oh. For instance the old the Sphinx. We found it was an open SSL library. We chose to release it in May 2006. It's for networked devices we have security functions but our guess what's what. Why choose this
 library because these library too old to be over in the real world to believe that the app is divine Viens because all this library open this to sell a library will be found in commercial product. Wasn't there some before wasn't a similar semantic to see which was in 2007. So as the is like old school really. Next thing related to Operation system configuration is hoodoo. And it's actually everywhere and actually everywhere for management interfaces and put in the web services shell et cetera and it's implemented in terrible way as you can see type and WD data have all ability to execute all command. And some scripts just execute the four execute any common through the pseudo what is why. If you have any small vulnerabilities in the web interfaces like in this case it's a common injection you can execute a whole month with Sudo. So it's again nineteen nineties. Next point it's software not system but software design point of view. It's this from software design. But a lot of open source components which implement the IP stack now are all think. But for management perspective we use mostly HP and things around it. So let's analyze it CTP and what management interfaces. So in this case it's not so old school like a system aside everything and base it on. No Jesus and JavaScript. It's like very cool but under the code you can find hard code Meeks from the Perl Java peach pea whatever which like I dunno looks like a guy's developed with the last 10. Yes we've all this more than no Jesus stuff developers confuse the client and the seller because you know javascript on both sides and it's hard to understand. But as several of his clients said I will show you examples and Navarro a lot of simple things like a slow ish repeat those attacks which should be fixed it for a long time ago but still you can stop a web interface if you issued a request so few examples about a client site. JASON see surfers everywhere so almost more web interface implement the protection from cros
s-eyed requests forgery in proper way. Exercise is everywhere and this is not a problem. So as a response from the product manager of one vendor they told me that exorcists from process scripting for their application that's not the issue because chrome blocks it. It's just an example of using that success on site of such appliances in this appliance so we can use a combination of the X SSN cross-eyed requires forgery to download and upload certificate which use it throughout the difficulty of the server which control plan. If management server is just one that should be a request and obviously there is no response from the vendor we just silently fix it so we decide to publish it for the full discussion. One more example of the perfect golf indication. So in this case you can see what this client side JavaScript which just send requests to the server to log in status function and if a user is what it is go to the request page. In our case go to the user name and password page. So these 100 percent client site more with checks on the server side justice just if you can change it. Second the example is just perfect. So guys I think he tried to port them. Authentication from the server side. The client said I understand what this javascript which seeks. That's still javascript but it doesn't work in browser and he just like commented and say if user name is with and password is this then go home. So off indication this past so this highlighted inbox. So with all of litigation on these books. Next finally things which are related to his divine is about different privilege escalation if you're already able to get access to any of vehicle appliances site. You can try to establish connection from this appliance to other appliance foods and the local cost function. So while it can be interesting because there are a lot of open source components for the remote management for instance I like challenge the box which provide you like shelf who's a web interface or mooning or s
olar which like system management boxes we just indeed two Loko cost. So you cannot establish connection from outside because it's listening to the local host. But if you're already on these boats and you can connect from these boats to local host when they establish this connection works and these words because on each vehicle on each appliance you have a lot of water the appliance which still listen to one IP addresses if you have experience with the talker for instance. So all Dockers container have all my IP address but from the all code for all computer it's actually connection from local cost. So if you own any of the digital appliance you can own next all the toll machines installed here. This gif highlights different interests in ways to escalate privilege inside the box. For instance if you are able to let me switch to the laser pointer you can see it. OK. If you can get access to traffic processor for instance for the some DCP stakeholder abilities next you can get access to management application and this management application in most cases have no any traffic filtering and trust to management application of all we're told network functions which run on this appliance. So you can do horizontal privilege escalation or next jump down to operation system level and then just go to the management plan upstairs to the management of places but it's it's really boring to find work application vulnerabilities in such big amount of code which is why we just download this quote from the different Network Appliance and drop it into the interactive code analysis system. In this case we use positive technologies application inspector and this help us to find a lot of faith in our abilities including such funny things like for instance poorly patch with vulnerability in the Citrix is the one which was budget in 2017 but it still works if you use not get it she could be this quest but post she could be request so way way way patched it's once again in risk in this case 
but then also you'd see your this time your adolescent style vulnerability. So it's obviously patch traversal but it's just reminder that attachment sleep to the jealousy and the shadow of grid that is so if you sent attachments shadow you can get shadow file. So this a full list of a full list of vulnerabilities we found during just source code analysis without the you know brand interaction. Next step is crypto because the security appliances shouldn't implement cryptography and Vera too thinks it's SSL Tillis and type Sec. In most cases SSL has use it to protect management interfaces between the different appliances and because it's automatic it's use different kind of automated setup. We found that there are a lot of things related to the unsecured configuration. For instance we can use as assertively as we roll forward secrecy. So if you have access to the certificate you can sniff of traffic you can give out full mentions a middle different the things related to the old cyber feuds like tribal dress or or C4 for IP sake. We found that in most cases we used a very strange way to select a certificate or their appreciated keys which in most cases just color coded. So one example from real life our example we will publish soon from against Citrix not scalar. All these appliances use master control not protocol to communicate between the orchestration plan control plan and data plan. It's right around on the DCP 2 1 5 6 and you said to us we were out for perfect secrecy and total interest in a certificate located in the home that towering user certificates and account not WWE data have full access to this certificate for some reason I don't know. Okay. Should maybe a read. But way to right. And what interests in all is demand appliances to be able to find during our security assessment. We use the same key payer which are located in appliance ageetc. My pinky so all as you run appliances in the vault you use to protect communication between management and chain. Sa
me key pair so if you know keeper and it's obviously you can just cut this from the file system you can passively or actively sniff traffic. The man in the metal spoof management complaints. And if this device have any application vulnerability you can override it. I don't know why but maybe if next time turn waiving change fees certificate. You can download old and two men in the middle of the game. Interesting stuff we found. We ran some tests from the dataset. Those attacks and found out about Saudi custom this use it in as the one appliance. Other ideas. Is vulnerable to record stores. So it's the old story. Some regular expression can spend a lot of your time. If you send specific queries it's always fix it in the default. SORRY CARTER Keith but still work and some more than as you want solution. And for sure if you do some in CA It's always work and give you some fun. Unfortunately we cannot present reverse engineer in part because most of such is the one solution we have for restriction in the light source in the agreement. Those are rare since January but just for fun. I think what some of engineers may also love Star Wars and haIf Marvel which is why insulation functions cause Marvel sucks. So just an overview of detected vulnerabilities. So green is good or bad so good for vendor but for us we are unable to detect it but then you can see that the most of classes of vulnerabilities like heart codes broken nexus control old products or Linux components or third party components were in most of such system. You can find most offsite system so just select any as divine and make shut interest interesting thinking is the 1 0 touch deployment so the Ft. It's a very cool feature. For instance let's imagine that you have a branch office you need to deploy to a branch office. If as day one it's absolutely not necessary to go where or establish telnet or SSA connection and try to upload configuration all you need is just to ship this device. Note this unique idea set
 up with the cloud console and ship this device to a remote location. This device will automatically connect to any Internet it see around connect to the cloud server load configuration and start duration. So for that for example how works in the Citrix system we have the supplies which ship it to the office. It's first tried to establish connection with the surrounded appliance. If no it's right to go to the 0 touch deployment service present all night. And next week's service will provide all configuration which you upload for the year as the one centre. So from a security perspective this scheme looks terrible. Why. Because this as you on a cloud deployment server should be friendly but any attacker. Not if you. No idea if you can brute force hisI.D. you can pretend to be this device. If you have any weaknesses in implementation of these management servers you can own all devices which deployed for his service and as you can see even Cisco which is like the best device from security perspective. We found on this product line let's say you have enough vulnerabilities it's and this you can see that the that patch provision in common injection vulnerability so its cloud server which to which all network device should connect sometimes but also now. Very funny things related to the distribution of this device because as it told you in the beginning most of such a device can be activated as a cloud appliance for that a double US or the other cloud services and we found that most of default default images use all the version we've known vulnerabilities. So you go to a W us you trust the vendor you activate with system and you receive attack because where there is no one that ability it's remind me and those story I'm really old man. Sorry. You know code red team. This kind of work kiddo. This kind of war of worms and this real disaster when your just start fresh Windows 2000 to which have Internet informations server by default and just connected into Internet to downl
oad patches you receive and the new infection and need to reinstall from the beginning. So these things look similar for me but is much worse because in this case is a security network device install it on the perimeter of your network. This overview of up to date statistics you can see what very few vendors that's actually not one of vendors have up to date version in the deployed on a double us or Internet. So I think it's abuse of the force but it's also a very interesting part of story as a security researchers we always work in the responsible disclosure way. And as you can see according to this article some vendors we also understand what a responsible disclosure is very important to communicate with community to fix issues. And we even have product Security Incident Response Team in place. Great but when we tried to submit vulnerability a report to this vendor we unable to find the email of this product Security Incident Response Team. So there is no pool. We tried to google for a different way. No luck. But we found what guys who did similar research before when they found that great. We sent e-mail to SEO or this company. Unfortunately my googling fu is not good enough but they are unable to find the e-mail. But I found that this guy on the LinkedIn and he hit handsaw within a few minutes actually and put me in contact. So if you tried to deal with as the one vulnerability report and just this way. So the prepared this table about different vendors how we communicate with researchers and you can see what actually a Cisco Citrix and the Lookout which actually obey and where but no. Not bad but all the rest. It's it's just a beginning. This is my favorite e-mail from the one vendor when we sent notification where start to ask Min why do we send this email from the g mail. Do we have a future idea what they mean. I need to present my passport for whatever it to submit vulnerabilities but the funnest fun finest things here this one the fraud with their is not g
eneric web service which have full access for Internet explosion. So after reading this Samarra go sleep and during the night someone told so to understand fretless cape of his divine rebuke to the bunch of script which works on the on the top of the different search engines like census showed on Google and you saw also and map lore scripting platform to fingerprint. Different as divine solution. We have a article published as do internets census and to which show and also some tools which can be read. Beautiful maps. If you want to present on the couch communication Congress because in our case it's useless. And to what we found. But there are not so many as divine devices yet. Is the Internet about free files and management interface which contains no wonder abilities and you can own it in a few minutes. And also we built some kind of vulnerability assessment tool which I hope you to find no vulnerability is seen as divine devices. This example or for openness DSH patch level as you can see what like some series from 2010 2014etc. This open source so you can find it on the top. We have two version. One is you one harvester which use Google showdown and sends us to collect information in all the all the all worlds Internets and also as one infiltrated infiltrator sorry which is branch of the network and maps crap pension scripts and you can use it during penetration testing so it's not necessary to be connected to the Internet. You can just use it inside the network. When we did this research we also found interesting article from silence about the Dark Web market. When the web guys sell user names in password to different Network Appliances let's say enterprise level Network Appliances and they found what varies whereas some IP addresses which we found during our assessment. Internet harvesting in this list and in our experience there is no such thing as a lark. We tried to find while such appliances can be so easily hacked and obviously the default password which
 hardcoded sometimes but never change it is use it was use it in these appliances. We tried to reach them and say guys maybe it's a better idea to use. Yeah. For instance it's hard coded as an MP not hardcoded by default. Community like public and again public for read write but we're told that spot SMP is off by default. But still simple. Short on search showed that more than 200 users of this as one way enable this SMP service and still use default password. So we have a lot of tools of each publish it in our github and the police contribute. There are a lot of things to do. We have the new fingerprints for is divine harvester and a traitor for his divine threat landscape description. We have new vulnerabilities and also it's like special CCC release. We start to publish motorsport models for that as D1. So we have public vulnerability description. You can create on models for it and conclusions. So from my perspective how as one development life cycle works. So someone comparably brilliant idea okay let's build one because God told us what it's like brilliant to have in a in the cloud. So what we have to do we can download a bunch of open sources are put off together sit up defiled and thinks and after all use it that as divine so as divine is a bunch of open source which not bad but still you need care about it and unit install patches configure in the proper way and maintain this complex products have problems with patch management have a lot of management interface like motion to mention and also a human dimension interfaces faces a lot of big defaults like Passport hardcoded certificate peace key keys for the IP sic and many vendors unfortunately have issues with the patch and responsible disclosure and this in the cloud. So if you decide to switch your network to that as the one hacking before by all you will fail. So thank you for your attention. I want to ask you to give become to that as the one you hope team to then is Maxime Nikolai. Nikita Alec and Ton
y who did most of the things here. Just like a fragment of his group. Thank you so much guys. We now have about like 15 minutes for questions and you know the rules. Police moved from Sunday and he goes on to say. Yep. Aside from kind of generic things that of any random living so we're going to have a better interface on this stage. Have you looked at any specific as well as security problems like the encapsulation of tunnels from stuff like that. So as do one you know what is more like technology like the one so 4 4 is the M but it's kind of protocol in which more or less we use it in different thunder in different solution and as to on every render implement thinks and all the way. As an example it's like Citic management protocol it says one thing we did here but we we didn't publish this for is for them. They're always Asian because again this V in their story very interesting because if you have X Heflin of ability in any crypto function next you can get access to our little machine. But again problem here. But there is no stand up and different vendors call a v and f so implement V and therefore differently. It can be QB m it's can be just a script which they upload to aeroplanes. So you're saying because everyone is writing their own code there are a lot of bugs to find. For people who put the work in. Yeah. Okay. Okay. And you can just like it sort of set it to try to buy things for the Ebi to fuck to hock it. You can just go to a doubleU.S. and activate it for free. Okay Mike one may well be simple there was a lot of vendors that you were locked in. What about Juniper is the one who did not know yet. Looking forward we're gonna trail investigation. Thank you for your time. And yes please. OK. Say thanks for the walk. You mentioned a lot of these virtual machines were like running in apartment buildings or something holding. What are they running is like Kensington mob cash levels are usually all these criminals. It's like always 2 6 or something like that.
 No not not always two point six It was like the worst example so some census song some of us like you knew. So it's not necessary again for natural functions for authorization someone vendors they call the IMF just a bunch of script with download and change configuration of default sets. The sensor and this case we can use very old to kernel either way more or use more recent version. This is so yeah yeah it's okay. And the hard hitting how was it done. There are other pollutants firm's executive record in most cases. No way so no. Our polymer are more or things like this to no jets security or ceiling looks nothing like. All right. Good question. Have you looked in to Cisco. Nova for Cisco. Sorry it's it's night. Find the least tin din din for Cisco. We did it our exercise at the lip Taylor all right. Last question. Oh actually just a place. Maybe you drop the slide in with the 90s called they won. But also it's back. Sorry. Because there were so many hard coded passwords everywhere. Maybe you should just drop a slide in the 90s called they want their credit passwords locking in tie them. So it takes public thinks. Was that a question of just recommendation. All right. We have a few minutes left but if there are no questions left then I would call it a day. So another warm round of applause for that.