It's actually books 4 and 5 (not books 3 and 4) that you want to read together as one big book. The reason is that book 4 focuses on one set of characters, and book 5 focuses on a different set of characters, but they take place at the same time. It's a weird construct, and they make more sense and you can understand the timing of events better if you read those two books together. There are published chapter lists on line that tell you the best way to do it. It basically means you read like 5 chapters in book 4, then 5 or so in book 5, then 5 or so in book 4, etc. Trust me, that is the best way to do it.
Book 1 and the first season of the show are very closely aligned, though I will say that there is a key mystery that is hinted at in book 1 that has not been dealt with on the show yet. It is fairly subtle in the book hints, but the show ignored it for the most part. Mainly that is because the biggest hints come through in flashbacks to earlier events, and the show did not do those flashbacks. It would be hard to show them on TV without giving away the mystery. If you want to know what the mystery has to do with, it is:
Book 2 and season 2 did start to diverge more, and the divergence has grown larger with each season of the show with the biggest divergences happening last season (season 4). It is also fairly obvious that season 5 is going to diverge wildly from the books and they may end up being two different stories in the end because the show may not wait for GRRM to finish the books and is going to pass him up here soon. In fact if they do not release book 6 before next season of the show, I may wait to watch next season until after I get a chance to read book 6.