We've mentioned grappling rules before. They are by far the most notorious example of overcomplexity ever devised in a game system. Not necessarily the most complex - <cough>GURPS Vehicles<cough> - but certainly the most notorious.
Of course there are some gamers who treat such as a challenge. Nay, a test of manhood.
Transcript
Pete: Anakin, three welding arms swing at you.
Anakin: I take a... what's the term? Poise action? To trigger attacks of opportunity?
[SFX]: 13!
[SFX]: 20!
[SFX]: Whooom!
[SFX]: K-zzrssshh!
[SFX]: 16!
[SFX]: Whooom!
[SFX]: K-zzrssshh!
Pete: Wow, that's pretty good for a...
Anakin: What?
Pete: Um, you know... Anyway, Padmé! A flying creature lands next to you and tries to push you off the conveyor.
Padmé: I hit him!
Pete: You have no weapons.
Padmé: Oh. I better... uh... grab him then.
Pete: Awesome. Fortunately I have the grappling rules summarised in these handy tables and flowcharts.
[SFX]: <crinkle crinkle>
[SFX]: <crinkle>
[SFX]: <crinkle crinkle>
Pete: Sally, can you move your drink?