There is no need to keep the downloaded files and they can be deleted every week or so..
You can also configure this delete task in Patch Manager Administration tab --> Data Pruning option ( As shown in below screenshot. )
Prune download records
Records older than (days): Specify number of days you would like to keep the files in download folder. (As shown in below picture)
For Example : Below screenshot shows files deletion is configured every 20 days.