Attachments are downloaded into the specified folder (by default, /download) as needed. This folder is under your full control, you can remove unneeded files at any time.

When using POP3, attachments are also saved into a hidden folder on the phone’s internal storage during scheduled mail checking, this is called caching. A cached attachment will be moved into the /download folder without having to download it from the server.

When using IMAP attachments are loaded as needed. There is a setting under Settings > Manage Accounts > Options > Account options to preload and cache attachments up to a specified limit (set per message).