Update - I think I have found out what it actually is
After extensive testing in isolated/quiet roads near my house (had to go do this in the night so there was no traffic).
1. First of all, to give a background, I generally baby my vehicles most of the time. I do open in up once in a while depending on the open road and my mood etc.
2. So I give very little gas when driving (because of my babying habit), which means that most of the initial gear shifts maybe upto 4th gear happens with the RPM less than 2000
3. Under the above conditions, I see that the engine is revving/increasing RPM to accommodate the gear change since it is too low (I can see the RPM go up and then down during the gear change). this automatic blip in RPM is the noise I hear which is basically the engine rev
4. So I decided to test it by driving like a normal human being and gave sufficient gas which made all the shifts at or over 2000 RPM, in this scenario, I do Not hear that noise (that hmmm kind of noise) so basically meaning that the engine has sufficient RPM to accommodate the gear shift without have to rev it any further.
I think this is what it is, if you have any insights from your end, I would really appreciate it.