You are approximately 25lbs underweight for your height. You can take care of your body as a vegan; however, you need to be vigilant in eating the combination foods and eating enough food daily so your body gets the protein, and nutrients it needs. As a vegetarian you should be getting all your vitamins and minerals from the foods you ate except Vitamin B12 and maybe Calcium (if you have a history of bone disease in your family). Vitamin B12 supplementation you need to take because it comes from meat sources and it not available through vegetarian diet. Calcium supplementation may be needed if your intake is on the lower side depending on what you eat and your bone density. Soy products are a good source of calcium. Eat non-meat sources (soy products, nuts), vegetarian combinations (rice & beans, pasta & beans, soy cheese & pasta, salads with nuts & seeds), substitute dairy products (soy milk/soy yogurt), use plant oils, non trans-fat margarines, and eat a lot of fruits and vegetables. Eat daily high fiber breads and cereals (oatmeal). I recommend you read up on vegetarian and combinations of starches / vegetables and beans / nuts for protein, calcium, and other nutrients. Good sources of information are the American Dietetic Association website, www.eatright.org and Vegetarian Resource Group website, www.vrg.org
Just my recommendation from studying nutrition, you probably need to find a nutritionist close to you and find out if your body is really getting everything it needs. A general physical probably won't be able to really help you with that. Having meat in your body is healthy for you, but I would make sure to buy meat with no hormones or steriods. I personally wouldn't avoid it all-together though. If you're 5'5 and 98lbs, I believe that are underweight and that can affect your iron, protein, potassium levels, low vitamin levels, low blood pressure - which you probably already have - are you getting legs cramps? i know that seems random but for people with low blood pressure - they are tired, fatigue, poor circulation which sometimes causes leg cramps.
Make sure you're taking vitamins - they need to be whole food vitamins, that is VERY important, otherwise if you're buying regular cheap vitamins, you're basically eating chemicals and chalk. Mega Foods is a good brand that makes whole food vitamins, you will get SO much more nutrients for your body. It makes a big difference. They are more expensive but totally worth it, especially with your diet.