A clear and comprehensive picture describing the physical processes which regulate the stellar mass assembly is still missing in galaxy formation scenario. Stellar mass assembly in galaxies is believed to result from several physical processes such as star formation from in-situ or accreted gas, major or minor mergers, as well as supernovae and AGN feedback. The relative contribution and operating timescales of these different processes are still a matter of debate. I will present some observational constrains that we established using a large and deep sample of 200,000 galaxies in the COSMOS field. Our analysis is based on precise 30-band photometric redshifts and new NIR imaging data. I will show how these data help us to constrain the evolution of the stellar mass function, the galaxy star formation history and the efficiency of quenching processes out to z~4.