I want to overwrite WP Core (wp-admin, wp-includes, twenty-x themes and so on), may I?

No you may not. We run, secure and update all WP Core files. Do not modify any WP Core file, it will be automatically overwritten.