Is it possible to remove the default "Home" and "My Access" tabs from the default access for everyone?
To explain how the left side task list works, each task (i.e. Create Users, Modify Users, etc.) has a category assigned to it within the properties of the task. You can see this by going to Modify Task and selecting one of the tasks. On the Profile tab of the task you will see the category. So "Home" and "My Access" are both categories that are assigned to multiple tasks. There is no way to directly control those categories. But, the tasks that are listed under those categories can be "turned off" or "turned on" depending on the role associated with those tasks. So for example if the task "Modify User" has "RoleA" assigned to it and your user is a member of "RoleA", then that user will see the "Modify User" task. However, if your user is not a member of "Role A" then they will not see that task.
So to answer your question, if your user is a member of a role that has access to a task that is in a particular category, then that category is going to be displayed for them. If your user does not have access to any tasks in a particular category, then the category is not displayed. So if you wish to remove that category, you must remove that users access to each task listed under that category. You can do this by either modifying the roles associated with the individual tasks, or create new roles for your users and associate only the tasks you want to show with those roles.