There are three unsolved problems in thin film lubrication (TFL) since it was proposed 20 years ago, i.e., the determination of the type of molecules that can enter the contact region efficiently during sliding, the orientation of molecules in the contact region, and the effect of solid surfaces on the liquid molecular orientation in TFL. In order to answer the first two questions, an in situ measurement system comprising a self-designed Raman microscopy and relative optical interference intensity (ROII) system was set up to study the molecular behaviors. A variety of binary mixtures were used as lubricants in the test, and the concentration distribution profile and orientation of the additive molecules in TFL were characterized. The molecular behavior was determined via a combination of shearing, confinement, and surface adsorption. Furthermore, the difference in molecular polarity resulted in different competing effect of surface adsorption and intermolecular interaction, the influence of which on molecular behavior was discussed. Polar additive molecules interacted with the steel surface and exhibited an enrichment effect in the Hertz contact region when added into a nonpolar base oil. No enrichment effect was observed for nonpolar molecules that were added into the nonpolar base oil and polar molecules added into polar base oil. The enrichment of additive molecules enhanced the film-forming ability of the lubricant and resulted in a reduction in the friction coefficient of up to 61％. The orderly arrangement of the additive molecules was another reason for the friction-reducing. A binary multilayer model was proposed to illuminate the molecular behavior in the TFL, and the model was supported by contrary experiment results in elastohydrodynamic lubrication. This research may aid in understanding the nanoscale lubrication mechanism in TFL and the development of novel liquid lubricants.