First of all, to configure Umbraco to show URL's without the .aspx extension, you're going to need to locate the 'umbracoUseDirectoryUrls' setting in the ' appSettings' section of the 'web.config' file for your site.

<appSettings>
<add key="umbracoUseDirectoryUrls" value="true" />
</appSettings>

Once you've opened this file, all you need to do is change the 'umbracoUseDirectoryUrls' value to 'true'.