I did some google-fu on why the gap between the invention of the saddle and the stirrup was so long.
Here is the answer I found from r/askhistorians -
You need a certain saddle design to make a stirrup work. It needs to have a tree (a firm wooden frame like this) for the stirrups to anchor to, otherwise every time you put weight in one stirrup, the saddle will slip like this. The lack of a tree also creates abnormal pressure points on the horse's back, which will be greatly worsened by the stirrups. That will cause the horse back pain, which leads to an ornery horse that's apt to buck you off (or worse, become crippled).
Most early saddles served only to make riding more comfortable for the rider, so they were treeless. The first saddles to have trees were the Roman saddles that Agrippa911 linked to (those saddles could easily have stirrups attached; it wasn't too long after the invention of those saddles that stirrups came into use).
I should note that nowadays we have treeless saddles that do not cause pressure points on the horse's back and and can have stirrups attached, but they were developed very carefully by vets to avoid putting too much pressure on any one point in the horse's back. They also use elastic to create a tighter girth, preventing saddle slippage.
Picture of a saddle tree:
