Kittens are unable to pass urine or faeces without assistance for the first few weeks of their lives.
You will need to stimulate the ano-genital region, the area around the bottom and the urinary opening, with a damp piece of cotton wool before and after each feed to encourage toileting. The mother would normally do this by licking the kitten's rear end before, during and after each feed.
Try to do this in the litter tray so that kittens associate the feeling of litter under their feet with going to the toilet. From about four weeks of age, placing the kitten in the litter tray should encourage them to pass urine and faeces on their own. Ensure the litter tray has shallow sides so the kittens can easily access it - and provide a litter with a sandy texture.
Constipation can be a problem for hand reared kittens. Normal faeces has the consistency of toothpaste. If the faeces becomes very hard, making the kitten strain excessively, or if a kitten does not pass any faeces for a day, speak to your vet.
Diarrhoea can be caused by overfeeding, by giving an overly-concentrated milk replacer, or by an infection. Because a kitten's condition can deteriorate rapidly, talk to your vet if your kitten is suffering from diarrhoea.