Corn Snake is scared of me

Hi everyone! I recently bought a brand new baby Albino Cornsnake 3 weeks ago. Akai is 3 months old now! After the first week of leaving the snake alone in its cage to allow it to settle in, I have been handling him outside of the cage for some time everyday to help him be tame. So far so good. However once he is back in his cage. It feels as if he has forgotten that he isnt scared of me (since while outside he is constantly flicking his tongue, looking around, and exploring the new environment and not showing any signs of being scared), and BOLTS to the nearest hide. This is very confusing for me to understand because it feels as if he doesnt like me. Even if he is out drinking from his water bowl, and I walk into the same room, he will flinch back very quickly even though im on the other side of the room and BOLT back into his hide. I feed him frozen thawed pinkies every 6 days, and hold him every day for around 30 minutes. This excludes the two days I give him to digest his food. He has everything he needs and I am not sure what is causing him yo be afraid of my presence. Any info would help! Thanks in advance -Akai