That's true, which is why I didn't really want to mount to the cupholder.
Your method might work, but unless that bracket 1) didn't flex and 2) wasn't very thick, you'd have issues with the cupholder insert sitting flat, and it wouldn't be able to fit in the rear position at all. That, and the two mounting bolts under the rubber mat are positioned near the center. So you'd essentially be creating a sideways U shape (including the armrest), which isn't conducive to a stable bracket. I suppose you could double side tape it down to the back to be stable enough.
I am considering redoing the wrapping on the armrest because I want to put red stitching on it, and actually take the time to sew it in the right shape. But I suppose I could do that later.
Now an idea would be to put an L bracket on the cupholder insert, so you could mount the hinge at the exact same place as it currently is, then hard-mounting the cupholder insert into the center console (to the bottom for instance). Those holes you put there could easily be covered up with the rubber mat later if you decide to remove it. You'd just have to cut up an old mouse pad to create a new rubber pad if you don't want holes drilled in your current one. And replacing the cupholder insert of course. In fact that idea sounds pretty good right now. I'll take a look at it closer... hopefully I have time today.
I commute to work for a long time each day so a lot of my time is spent in the car. Where I live up north, I don't have a garage, tools, etc. So the only time I can work on anything like this is down south near my work and family, which has the garage and tools, and after work at that.
Just in case you were wondering why I am somewhat rushing this (the supplies for this already sat in my trunk for two weeks D: )