User without any of the Page access right, but with Dashboard access rights, can access a custom page that is setup for specific group access right and the user doesn't belong to that group. Is that a defect or by design?
This is expected by design. The Dashboard access rights are given for the 'Page' Object. Therefore, the instance or global access of 'Dashaboard' is to navigate, view, edit, delete "pages".
The 'Dashboard' rights are meant to be given to users that you want to create pages on the application side, not on the administration side because it does not include administration access rights whereas the 'Page' access rights includes administration access to edit and manage pages from the administration side.
It is best to use the 'Dashboard' rights for specific page instances instead of using the 'global' access to ensure they only can edit the ones that the administrator wants that person to edit.