2020-01-14: What's the probability of a downloaded ZIP being broken?

I found a fun question in my inbox today: If an application downloads a ZIP file with an update, what is the probability of the ZIP being corrupted? And should the update's hash (e.g. SHA256) be always attached as well? Let's take a look at the details.

There are basically two parts to the answer - the probability itself, and best practices.

Starting with the first one, let's consider a typical "download stack", i.e. HTTP over SSL over TCP over IP, and the ZIP file format itself. There are 3-4 mechanisms in play that need to be considered here:

  • TCP packet's checksum: It's a 16-bit value, meaning (and slightly simplifying the problem to a rule-of-thumb) that if the transmitted data gets corrupted, there is a 1/216 (or ~0.15%) chance of the corruption not getting detected by the checksum. In practice if you're transmitting a lot of data (e.g. 1 GB) through a noisy medium (e.g. some form of radio), you're basically guaranteed to run into this problem.
  • SSL/TLS (H)MAC/AEAD: Long story short SSL/TLS tries to do its best to protect the payload from being corrupted on purpose by a third party. Depending on the version this was done by either calculating a 128/256-bit MAC of the data (i.e. hash-based Message Authentication Code), or using AEAD (Authenticated Encryption with Associated Data). In general, it can be assumed that either approach will detect accidentally corrupted data, i.e. the probability of a corruption accidentally colliding a hash is basically non-existant, or 0.00000000000000000000000000000000000029% for 128-bit MACs, 0.00000000000000000000000000000000000000000000000000000000000000000000000000086% for 256-bit MACs, and so on).
  • ZIP's CRC32: A 32-bit value. Not cryptographically safe (actually quite unsafe in fun and interesting ways, but that's for another time), but it still should be able to detect most corruptions that happen to file data in the ZIP archive (but NOT to ZIP headers and e.g. file names; even though file names are in two places in a ZIP files, almost no ZIP extractors compare both file names against each other).
  • In addition every protocol parser (and the ZIP parser) on the way might detect corruption in the headers (though this isn't guaranteed). Also lower level protocols (e.g. Ethernet's FCS) might detect some corruptions - they usually also use 16- or 32-bit checksums.

So in the end, if we use HTTPS we should be safe, at least from corruptions made during transit (most of them are). However, if a corruption would be introduced e.g. while the data is still being handled on the sender's side (and a cosmic ray would hit the CPU in the right place), then by the time the data is safely transmitted through SSL it's already too late. So an additional update hash would save the day.

What about current best practices?

Basically it's recommended that downloadable updates are cryptographically signed with a private key, and after the download is complete, the application checks whether the signature is correct using a public key (that's hardcoded in the application). This way apart from detecting accidental corruptions, we're also stopping a potential attacker from supplying their own update package (e.g. after hacking the update server). Of course this means now we have to protect the private key and somehow safely incorporate it into our build process, but at the end of the day it's probably worth it.


Read more... [ 1 comment ]

2019-10-13: Wyzwanie o wejściówkę na Security PWNing

W poniedziałek 14.10.2019 o 9:00 rusza wyzwanie z prostymi zadaniami ala CTF/hackme/wargames o darmową wejściówkę (oraz kody rabatowe o rosnących nominałach) na konferencję Security PWNing 2019.

Adres strony z wyzwaniem: https://gynvael.coldwind.pl/pwning2019/

Na wyzwanie będzie składać się 5 poziomów o rosnącym poziomie trudności (od bardzo prostych do średnio trudnych, przynajmniej z mojego punktu widzenia *wink*):

  • Po przejściu 1 poziomu uczestnik otrzyma kod rabatowy na 150 PLN od ceny netto konferencji (wielokrotnego użytku).
  • Po przejściu 2 poziomu uczestnik otrzyma kod rabatowy na 170 PLN od ceny netto konferencji (wielokrotnego użytku).
  • Po przejściu 3 poziomu uczestnik otrzyma kod rabatowy na 200 PLN od ceny netto konferencji (wielokrotnego użytku).
  • Po przejściu 4 poziomu uczestnik otrzyma kod rabatowy na 250 PLN od ceny netto konferencji (pięciokrotnego użytku - kto pierwszy ten lepszy). [nagrody odebrane]
  • Po przejściu 5 (ostatniego) poziomu uczestnik otrzyma kod rabatowy na 100% ceny konferencji (jednokrotnego użytku - kto pierwszy ten lepszy). [nagroda odebrana]

Kody się nie sumują i działają jedynie do końca października. Sam konkurs trwa do 20 31 października (włącznie).

Podejmij wyzwanie! Zgarnij bezpłatną wejściówkę lub dużą zniżkę na bilet! 4 edycja Security PWNing Conference 2019, 14-15 listopada, Warszawa. Konkurs trwa do 31 października 2019. Zapraszają: Instytut PWN oraz Gynvael Coldwind

Czytaj dalej... [ 0 komentarzy ]

Five newest or recently updated notes (these are unfinished posts, code snippets, links or commands I find useful but always forget, and other notes that just don't fit on the blog):

Click here for a list of all notes.

EN Security papers and research notes

Some conference slides are linked at the bottom of this page.

EN Selected vulnerabilities

The full list of vulnerabilities discovered by me (including collaborative work) can be found here (please note that the list might be out of date).

The Google Application Security / Research site might also contain some of my findings.

EN Coding (selected posts)
EN Tools and libraries
  • PiXieServ is a simplified PXE (network boot) server for Windows and Linux-based OS, created for testing of very small home-made OS. See also the post about it.
  • ExcpHook, a system-wide exception monitor for Windows XP 32-bit. Useful if you're fuzzing something that doesn't like having a debugger attached.
  • Ent is an entropy measuring tool for reverse engineering reconnaissance (see also a post explaining how to use it).
  • HiperDrop is a simple command line process memory dumper for Windows, with a few different work modes.
  • asmloader - this little app executes headerless machine code (compiled assembly code). It's meant to be an aid in learning/teaching and playing with assembly, as well as the right tool when you just need to execute some machine code.
  • NetSock is a simple socket/networking lib/wrapper for C++ I've wrote back in 2003 and update from time to time - I use it for most of my network-enabled projects.
PL Videotutoriale i podcasty [ 0 views | 0 videos | 0 subscribers ]

Subscribe to me on YouTube W wolnym czasie prowadzę videocasty na żywo o programowaniu, reverse engineeringu oraz hackingu/security:

Livestream | Kanał na YT | Archiwum starszych odcinków

Najnowszy odcinek: Gynvael's Livestream #75: Implementujemy serwer FTP
[ 0 thumbs up | 0 comments | 0 views ]

Dodatkowo: ReverseCraft - starsza seria podcastów o reverse engineeringu i assembly.

PL Edukacyjnie (wybrane posty)

Dla programistów:

Security / hacking:

  • Hacking - jak uczyć się security/hackingu i spać spokojnie.

Dodatkowo, kilka przemyśleń na temat odnajdywania się na rynku pracy w IT:

PL Programowanie (wybrane posty)

← trochę więcej postów jest po angielskojęzycznej stronie.

PL Gamedev i GFX (wybrane posty)

Grafika generowana proceduralnie:

【 design & art by Xa / Gynvael Coldwind 】 【 logo font (birdman regular) by utopiafonts / Dale Harris 】