As Sylvia says, you could break your hymen by being fingered. That said, the commonest definition would be that the man's penis has to go inside the vagina.
if you broke your hymen then physically you've lost virginity ~ but virginity is generally deemed to mean untouched or un-penetrated by a man.