2020-01-14: What's the probability of a downloaded ZIP being broken?

I found a fun question in my inbox today: If an application downloads a ZIP file with an update, what is the probability of the ZIP being corrupted? And should the update's hash (e.g. SHA256) be always attached as well? Let's take a look at the details.

There are basically two parts to the answer - the probability itself, and best practices.

Starting with the first one, let's consider a typical "download stack", i.e. HTTP over SSL over TCP over IP, and the ZIP file format itself. There are 3-4 mechanisms in play that need to be considered here:

• TCP packet's checksum: It's a 16-bit value, meaning (and slightly simplifying the problem to a rule-of-thumb) that if the transmitted data gets corrupted, there is a 1/2¹⁶ (or ~0.15%) chance of the corruption not getting detected by the checksum. In practice if you're transmitting a lot of data (e.g. 1 GB) through a noisy medium (e.g. some form of radio), you're basically guaranteed to run into this problem.
• SSL/TLS (H)MAC/AEAD: Long story short SSL/TLS tries to do its best to protect the payload from being corrupted on purpose by a third party. Depending on the version this was done by either calculating a 128/256-bit MAC of the data (i.e. hash-based Message Authentication Code), or using AEAD (Authenticated Encryption with Associated Data). In general, it can be assumed that either approach will detect accidentally corrupted data, i.e. the probability of a corruption accidentally colliding a hash is basically non-existant, or 0.00000000000000000000000000000000000029% for 128-bit MACs, 0.00000000000000000000000000000000000000000000000000000000000000000000000000086% for 256-bit MACs, and so on).
• ZIP's CRC32: A 32-bit value. Not cryptographically safe (actually quite unsafe in fun and interesting ways, but that's for another time), but it still should be able to detect most corruptions that happen to file data in the ZIP archive (but NOT to ZIP headers and e.g. file names; even though file names are in two places in a ZIP files, almost no ZIP extractors compare both file names against each other).
• In addition every protocol parser (and the ZIP parser) on the way might detect corruption in the headers (though this isn't guaranteed). Also lower level protocols (e.g. Ethernet's FCS) might detect some corruptions - they usually also use 16- or 32-bit checksums.

So in the end, if we use HTTPS we should be safe, at least from corruptions made during transit (most of them are). However, if a corruption would be introduced e.g. while the data is still being handled on the sender's side (and a cosmic ray would hit the CPU in the right place), then by the time the data is safely transmitted through SSL it's already too late. So an additional update hash would save the day.

What about current best practices?

Basically it's recommended that downloadable updates are cryptographically signed with a private key, and after the download is complete, the application checks whether the signature is correct using a public key (that's hardcoded in the application). This way apart from detecting accidental corruptions, we're also stopping a potential attacker from supplying their own update package (e.g. after hacking the update server). Of course this means now we have to protect the private key and somehow safely incorporate it into our build process, but at the end of the day it's probably worth it.

Gynvael

Read more... [ 1 comment ]

2019-10-13: Wyzwanie o wejściówkę na Security PWNing

W poniedziałek 14.10.2019 o 9:00 rusza wyzwanie z prostymi zadaniami ala CTF/hackme/wargames o darmową wejściówkę (oraz kody rabatowe o rosnących nominałach) na konferencję Security PWNing 2019.

Adres strony z wyzwaniem: https://gynvael.coldwind.pl/pwning2019/

Na wyzwanie będzie składać się 5 poziomów o rosnącym poziomie trudności (od bardzo prostych do średnio trudnych, przynajmniej z mojego punktu widzenia *wink*):

• Po przejściu 1 poziomu uczestnik otrzyma kod rabatowy na 150 PLN od ceny netto konferencji (wielokrotnego użytku).
• Po przejściu 2 poziomu uczestnik otrzyma kod rabatowy na 170 PLN od ceny netto konferencji (wielokrotnego użytku).
• Po przejściu 3 poziomu uczestnik otrzyma kod rabatowy na 200 PLN od ceny netto konferencji (wielokrotnego użytku).
• Po przejściu 4 poziomu uczestnik otrzyma kod rabatowy na 250 PLN od ceny netto konferencji (pięciokrotnego użytku - kto pierwszy ten lepszy). [nagrody odebrane]
• Po przejściu 5 (ostatniego) poziomu uczestnik otrzyma kod rabatowy na 100% ceny konferencji (jednokrotnego użytku - kto pierwszy ten lepszy). [nagroda odebrana]

Kody się nie sumują i działają jedynie do końca października. Sam konkurs trwa do 20 31 października (włącznie).

Czytaj dalej... [ 0 komentarzy ]

Five newest or recently updated notes (these are unfinished posts, code snippets, links or commands I find useful but always forget, and other notes that just don't fit on the blog):

• 2019-10-26: Windows 10 file type association
• 2018-06-18: PHP quirks/security/CTF tricks/tips
• 2018-01-04: Python 3 BrokenPipeError pipe test
• 2017-11-12: IDA PYTHONHOME wrapper
• 2017-10-22: Python String/bytes as big number and back

Click here for a list of all notes.

• DLL shared sections: a ghost of the past - research notes taken while revisiting the infamous DLL shared sections; also, a description of finding and exploiting a Cygwin vulnerability. See also this post and these tools.
• PHP LFI to arbitratry code execution via rfc1867 file upload temporary files is an attempt to bring a specific LFI exploitation technique to common knowledge. You might also want to check out a follow-up paper by Brett Moore.
• Exploiting the otherwise non-exploitable - Windows Kernel-mode GS Cookies subverted described the internal mechanisms of generating GS cookies in drivers and a way to predict them.
• GDT and LDT in Windows kernel vulnerability exploitation is another paper I've written with j00ru, this time about using call-gates in kernel exploitation - quite a cool method since it works even if you have only a 1-byte write-what-where condition.

Some conference slides are linked at the bottom of this page.

• A few Microsoft Windows privilege escalations and some DoSes patched by MS10-021 and MS10-011 - this research was presented on HITB Dubai 2010 and CONFidence 2010 (video available here).
• Well, actually a few more bugs in Windows kernel and drivers discovered during our Bochspwn research (see this post and this one) using our kfetch-toolkit. They were patched in MS13-016, MS13-017, MS13-031 and MS13-036.
• Adobe Reader 9.5.1 and 10.1.3 multiple vulnerabilities - 62 unique crashes, from that 31 trivially exploitable and 9 more potentially exploitable, 11 CVE's assigned (CVE-2012-4149 to CVE-2012-4160). Some of these bugs were fixed for Windows and OSX releases of Adobe Reader in APSB12-16.
• Adobe Flash had also quite a lot of fixes (around 60 CVEs assigned). Some details can be found in these bulletins (in random order): APSB12-27, APSB12-24, APSB12-22, APSB13-17, APSB13-14, APSB13-11, APSB13-09, APSB13-05 and APSB13-01.
• Contributed to discovery of multiple low-to-high vulnerabilities in Google Chrome (CVE-2012-2851, CVE-2012-2855, CVE-2012-2856, CVE-2012-2862, CVE-2012-2863 and some other) - some of these were mentioned in this post.
• A lot of bugs in ffmpeg and libav which resulted in 892 (sic!) patches in ffmpeg and 299 patches in libav (CVE-2011-3930 to CVE-2011-3952 and some other).
• Cygwin cygwin1.dll shared section local privilege escalation (demo video) - discovered while revisiting old-school classes of bugs (see paper above).
• Two minor bugs in PuTTY and aterm and rxvt found while playing with terminal control codes. Put here to create some illusion of diversity.
• Mozilla Firefox 2.0.0.11 and Opera 9.50 information leak, also midly affected Safair, Konqueror and some other products (CVE-2007-6524, CVE-2008-0420, CVE-2008-0894, CVE-2008-1573). A demo video is also available.
• A small but funny bug in Total Commander 7.01 - an FTP client gets attacked by the server, leading to a path traversal.
• And there were also these two: a local privilege escalation that required a USB stick of death, and a funny compiler bug.

The full list of vulnerabilities discovered by me (including collaborative work) can be found here (please note that the list might be out of date).

The Google Application Security / Research site might also contain some of my findings.

• Unicode fun: PHP preg_match and UTF-8 with analysis of what happens when you forget that UTF-8 is multibyte; and a String-to-Integer vs Unicode - yes, the standard 0-9 digits are not the only ones supported by programming languages or unicode (see also this post).
• Digging deeper into C/C++: internals of static variables initialization, a discussion with myself about NULL/nullptr, the curious case of int a=5; a=a++ + ++a;, and what happens when scanf/atoi/strtol get an overly large number. Also, a way to magically make a list of functions at compilation time, and a way to create naked functions in gcc/g++ (or rather pure-assembly functions with no additional prologs/epilogs).
• Low-level goodies: a Hello World in C without using any headers / library functions, a process that frees almost all the memory it owns, and embedding GDB in assembly source code.
• Do not do this at work kids - OOP in .bat and OpenGL in .bat.

• PiXieServ is a simplified PXE (network boot) server for Windows and Linux-based OS, created for testing of very small home-made OS. See also the post about it.
• ExcpHook, a system-wide exception monitor for Windows XP 32-bit. Useful if you're fuzzing something that doesn't like having a debugger attached.
• Ent is an entropy measuring tool for reverse engineering reconnaissance (see also a post explaining how to use it).
• HiperDrop is a simple command line process memory dumper for Windows, with a few different work modes.
• asmloader - this little app executes headerless machine code (compiled assembly code). It's meant to be an aid in learning/teaching and playing with assembly, as well as the right tool when you just need to execute some machine code.
• NetSock is a simple socket/networking lib/wrapper for C++ I've wrote back in 2003 and update from time to time - I use it for most of my network-enabled projects.

W wolnym czasie prowadzę videocasty na żywo o programowaniu, reverse engineeringu oraz hackingu/security:

Livestream | Kanał na YT | Archiwum starszych odcinków
Najnowszy odcinek: Gynvael's Livestream #75: Implementujemy serwer FTP
[ 0 thumbs up | 0 comments | 0 views ]

Dodatkowo: ReverseCraft - starsza seria podcastów o reverse engineeringu i assembly.

Dla programistów:

• Poradnik Początkującego Programisty - czyli jaki język wybrać, z czego się uczyć, jak samodzielnie rozwiązywać problemy i co dalej jak już się zna podstawy.
• Zmienne i stałe: wartość, kodowanie, reprezentacja - rozłożenie zmiennych na części pierwsze, od wysokiego poziomu abstrakcji, do kodowania na poziomie RAMu i CPU.

Security / hacking:

• Hacking - jak uczyć się security/hackingu i spać spokojnie.

Dodatkowo, kilka przemyśleń na temat odnajdywania się na rynku pracy w IT:

• Wybór studiów a szanse na rynku pracy - mój punkt widzenia na wybór studiów, rynek pracy, oczywiście w kontekście informatyki, programowania, itp.
• Spis sposobów na dokumentowanie swojej wiedzy, czyli co można wpisać w CV oprócz praktyk i papierka z uczelni.

• Trochę o PHP, preg_match i UTF-8, czyli o problemach związanych z niespójnym kodowaniem znaków.
• Kilka przemyśleń na temat intów, do tego kilka eksperymentów z wrzucaniem dużych liczb (większych od INT_MAX) w funkcje scanf/atoi/strtol, oraz różnice w wynikach w dzieleniu z udziałem liczb ujemnych w różnych językach.
• int a = 5; a = a++ + ++a;, ile wynosi "a"? Wynik jest oczywiście niezdefiniowany, ale jednak jakiś musi wypaść.
• Sposób na stworzenie funkcji naked w gcc/g++.
• Z przymróżeniem oka - programowanie obiektowe oraz OpenGL w języku skryptowym batch (aka .bat).

← trochę więcej postów jest po angielskojęzycznej stronie.

Grafika generowana proceduralnie:

• .S.k.y., czyli software'owy rendering heightmapy w trochę niekonwencjonalny sposób.
  
  
  
• SIN*COS, czyli co wychodzi z mnożenia sin(x) * cos(x) z różnymi współczynnikami.
  
  
  
• Graficzna interpretacja plików znalezionych na dysku - Art of file.
• Prosty raytracer napisany w Go, wraz z wersją unowocześnioną by mulander.