mika/Atmosphere
1
0
Fork 0
mirror of https://github.com/Atmosphere-NX/Atmosphere.git synced 2025-07-03 08:02:14 +02:00
Code Issues Packages Projects Releases Wiki Activity
4,168 Commits 44 Branches 84 Tags 34 MiB
b7ec64ea16
Commit Graph

117 Commits

Author SHA1 Message Date
Michael Scire
b7ec64ea16 fs.mitm: add and use memlet module to temporarily steal applet memory while building romfs images. 
Starting in 20.0.0, the browser needs more applet memory to function, so we can't steal as much any more.
Thus, we now steal 14 MB on 20.0.0+ instead of 40MB.

However, since this reduces memory available for custom system modules, we are adjusting to compensate.
ams.mitm's heap size has been reduced from 32MB to 12MB (recovering 20MB).
In addition, fs.mitm now uses a new mechanism for stealing memory from the applet pool while romfs is being built.

On net, we are compromising:
    * Custom sysmodules lose memory available to them.
        On 19.0.0/AMS 1.8.0, there was 30 MB available for custom sysmodules.
        Stealing 14 MB instead of 40 MB, we lose 26 MB of that. Reducing ams.mitm's usage will gain us back 20.
        Nintendo also appears to...use 4 extra MB, in 20.0.0, from my test homebrew.
        So on 20.0.0/AMS 1.9.0, there should be 20 MB available for custom sysmodules.
        On the bright side, on <20.0.0/AMS 1.9.0, I guess there will be 50 MB available for custom sysmodules now?
    * totk mods will lose the ability to...put every file in the romfs on sd card. There will be some unknown maximum filecount for totk mods.
        On the bright side, implementing the transient memory stealing should improve compatibility for some mods which strictly add files?
 2025-05-09 11:55:21 -07:00
SciresM
f2ee44da74
romfs/ams.mitm/pm: refactor to dynamically steal heap for certain games. (#2122) 
* fs.mitm: skeleton the use of special allocation in romfs build

* pm: add api for ams.mitm to steal application memory

* pm/mitm: okay, that api won't work, try a different one

* romfs: revert memory usage increases; we'll handle torture games case-by-case.

* pm/romfs: first (broken?) pass at dynamic heap.

I cannot wait to figure out all the ways this is wrong.

* Release the dynamic heap a little more eagerly

* romfs: animal crossing is also not a nice game

* romfs: fix issues in close-during-build

* romfs: zelda is a blight upon this earth
 2023-05-14 03:06:52 -07:00
Michael Scire
f0240db75a fs.mitm: mitm glue for font replacement, before I forget 2023-05-08 17:40:10 -07:00
Michael Scire
1f5ec68a5c ams: fix compilation with gcc 13 2023-05-07 03:36:46 -07:00
Michael Scire
f1132fbf5a romfs: push clobber fix missed by git add 2023-01-24 14:03:54 -07:00
Michael Scire
9cd57b6c61 romfs: reduce peak allocation sizes by enabling intermediate context frees 2023-01-24 14:02:05 -07:00
Michael Scire
7e6b369605 ns.mitm/fs.mitm: allow program specific web override for non-hbl, if present 2022-07-25 16:24:57 -07:00
Michael Scire
bbf22b4c60 ams: globally prefer R_RETURN to return for ams::Result 2022-03-26 14:48:33 -07:00
Michael Scire
dd78ede99f ams: use R_SUCCEED, R_THROW globally 2022-03-26 00:14:36 -07:00
Michael Scire
3579797309 sf: add new debug-interface-id in cmif out header 2022-03-23 09:10:50 -07:00
Michael Scire
64a97576d0 ams: support building unit test programs on windows/linux/macos 2022-03-21 17:21:44 -07:00
Michael Scire
47218f0da8 ams: overhaul fs.mitm romfs ownership, bump to 1.2.4 2021-11-04 13:38:41 -07:00
Michael Scire
5eabca7f04 ams.mitm: more romfs building space/time tradeoffs. 
This is needed for Animal Crossing 2.0.0, which has >99000 fucking files.

We now do several passes over dir/file tables instead of one pass,
doing entire hash tables before we touch dir/file tables. Thus we
no longer need to simultaneously allocate hash table and dir/file table space.

In addition, we now do repeated passes building a segment of hash tables
at a time, when insufficient memory is available. Similar is also now the
case for file/dir tables, we try 0x40000 work buffer and divide by 2
until we successfully alloc. We don't allow a work buffer <0x4000, for
write/perf reasons. If a game triggers that, let me know I guess.

Hard to imagine a worse torture-test for this code than animal crossing.
 2021-11-03 23:52:38 -07:00
Michael Scire
bd1bcdf52b sf: prevent emitting mitm/defer code unless server in process supports it 2021-10-11 19:01:27 -07:00
Michael Scire
a595c232b9 strat: use m_ for member variables 2021-10-10 00:14:06 -07:00
Michael Scire
1ab0bd1765 ams: replace most remaining operator & with std::addressof 2021-10-09 14:49:53 -07:00
Michael Scire
229651fe72 ams.mitm: avoid using std::unordered_map, it manifests exception code 2021-10-07 22:59:16 -07:00
Michael Scire
ffc143860b strat: use ams::Main() instead of main(argc, argv) 2021-10-07 17:44:54 -07:00
Michael Scire
6a53726833 strat: build sysmodules with -Wextra/-Werror 2021-10-06 23:22:54 -07:00
Michael Scire
77fe5cf6f5 ams: the copyright^H^H^H^H^H^H^Hmplex plane is the algebraic closure of the reals 2021-10-04 12:59:10 -07:00
Michael Scire
41ab4c2c68 stratosphere: use SdkMutex/SdkRecursiveMutex over Mutex 2021-09-29 22:52:50 -07:00
Michael Scire
ab81ed2795 fs.mitm: fix memory leak in romfs build (closes #1031) 2021-09-20 17:53:29 -07:00
Michael Scire
846cde52e0 fs.mitm: ns registers program index map info, not am 2021-06-28 22:43:56 -07:00
Michael Scire
a3dd445b32 fs.mitm: add OpenDataStorageWithProgramIndex support (theoretically, closes #1250) 2021-06-28 22:24:26 -07:00
Michael Scire
15381409dc fs: fix missed operation rename 2021-04-30 09:02:58 -07:00
Michael Scire
74e4e70053 fs.mitm: fix cache of non-current-process data storages (closes #1371) 2021-02-15 19:39:32 -08:00
Michael Scire
e87e146112 ams_mitm: update for new sf semantics 2021-01-20 23:39:31 -08:00
Michael Scire
094cede39e ams: use util::SNPrintf over std:: (size/linker improvements) 2021-01-12 02:59:41 -08:00
Michael Scire
bf55776241 fsa: *Impl -> Do* 2020-12-05 03:05:06 -08:00
Michael Scire
1d8744164e ams.mitm: provide different boot0-protection for custom public key 2020-11-14 03:37:51 -08:00
SciresM
9fde97cfdd
sf: Change interface definition methodology (#1074) 
* sf: Begin experimenting with new interface declaration format

* sf: convert fs interfaces to new format

* sf: finish conversion of libstrat to new definitions

* sf: convert loader to new format

* sf: convert spl to new format

* sf: update ncm for new format

* sf: convert pm to new format

* sf: convert ro/sm to new format

* sf: update fatal for new format

* sf: support building dmnt under new scheme

* sf: update ams.mitm for new format

* sf: correct invocation def for pointer holder

* fs: correct 10.x+ user bindings for Get*SpaceSize
 2020-07-07 17:07:23 -07:00
SciresM
3a1ccdd919
Switch atmosphere's build target to C++20. (#952) 
* ams: update to build with gcc10/c++20

* remove mno-outline-atomics

* ams: take care of most TODO C++20s

* fusee/sept: update for gcc10

* whoosh, your code now uses pre-compiled headers

* make: dependency fixes
 2020-05-11 15:02:10 -07:00
SciresM
3bc2d79384
PRODINFO: Revamp blanking/write disallow policy. (#913) 
* exo/fusee: hookup new prodinfo settings

* fusee: new scheme doesn't need FLAGS_DEFAULT

* fusee: fix c/p errors

* ams.mitm: completely revamp prodinfo backup mechanism

* ams.mitm: Implement revamped blanking/write policy

* strat: make early boot more debuggable

* exo: condense flag logic
 2020-04-22 16:22:14 -07:00
Michael Scire
3da0cda4ae ams: centralize system thread definitions 2020-04-17 01:06:07 -07:00
Michael Scire
6719abec65 hos::Version: rename enum members 2020-04-13 22:19:44 -07:00
Michael Scire
065485b971 os: refactor/rewrite entire namespace. 2020-04-08 02:21:35 -07:00
SciresM
87ec045a98
mem: implement most of StandardAllocator (#860) 
This was tested using `https://github.com/node-dot-cpp/alloc-test` plus a few other by-hand tests.

It seems to work for the case we care about (sysmodules without thread cache-ing).

External users are advised to build with assertions on and contact SciresM if you find issues.

This is a lot of code to have gotten right in one go, and it was written mostly after midnight while sick, so there are probably un-noticed issues.
 2020-03-29 14:43:16 -07:00
Michael Scire
0af2758fde fs.mitm: use new namespace types for saves 2020-03-24 17:50:36 -07:00
Michael Scire
37f7afb426 ams.mitm: greatly reduce memory requirements to build romfs 2020-03-18 00:07:19 -07:00
Michael Scire
93004be59e ams: finish stdio -> fs bindings for stratosphere 2020-03-09 03:58:02 -07:00
Adubbz
c7026b9094
Implement the NCM sysmodule (closes #91) 
* Implement NCM

* Modernize ncm_main

* Remove unnecessary smExit

* Give access to svcCallSecureMonitor

* Stack size bump

* Fix incorrect setup for NandUser's content storage entry

* Fix a potential data abort when flushing the placeholder accessor cache

* Fix HasFile and HasDirectory

* Use r+b, not w+b

* Misc fixes

* errno begone

* Fixed more stdio error handling

* More main fixes

* Various command improvements

* Make dispatch tables great again

* Fix logic inversion

* Fixed content path generation

* Bump heap size, fix CleanupAllPlaceHolder

* Various fixes. Note: This contains debug stuff which will be removed later. I was getting tired of having to cherrypick tiny changes

* Fixed placeholder/content deletion

* Fixed incorrect content manager destruction

* Prevent automatic placeholder creation on open

* Fixed List implementation. Also lots of debug logging.

* Removed debug code

* Added a scope guard for WritePlaceHolder

* Manually prevent placeholder/content appending

* Revert "Removed debug code"

This reverts commit d6ff261fcc.

* Always cache placeholder file. Switch to ftell for preventing appending

* Universally use EnsureEnabled

* Abstract away file writing logic

* Misc cleanup

* Refactor placeholder cacheing

* Remove debug code (again)

* Revert "Remove debug code (again)"

This reverts commit 168447d80e.

* Misc changes

* Fixed file modes

* Fixed ContentId/PlaceHolderId alignment

* Improved type safety

* Fixed reinitialization

* Fixed doubleup on path creation

* Remove debug code

* Fixed 1.0.0 booting

* Correct amount of add on content

* Correct main thread stack size

* lr: Introducing registered data

* Reorder stratosphere Makefile

* Move results to libstrat

* lr: Cleanup lr_redirection

* lr: lr_manager tweaks

* lr: Imrpoved path handling and adjust ResolveAddOnContentPath order

* lr: Organise types

* Add eof newlines

* lr: Eliminate unnecessary vars

* lr: Unnecessary vars 2 electric boogaloo

* lr: Various helpers

* lr: RegisteredLocationResolver helpers

* ncm: Move ncm_types to libstrat

* ncm: Misc cleanup

* Implement NCM

* Modernize ncm_main

* Remove unnecessary smExit

* Give access to svcCallSecureMonitor

* Stack size bump

* Fix incorrect setup for NandUser's content storage entry

* Fix a potential data abort when flushing the placeholder accessor cache

* Fix HasFile and HasDirectory

* Use r+b, not w+b

* Misc fixes

* errno begone

* Fixed more stdio error handling

* More main fixes

* Various command improvements

* Make dispatch tables great again

* Fix logic inversion

* Fixed content path generation

* Bump heap size, fix CleanupAllPlaceHolder

* Various fixes. Note: This contains debug stuff which will be removed later. I was getting tired of having to cherrypick tiny changes

* Fixed placeholder/content deletion

* Fixed incorrect content manager destruction

* Prevent automatic placeholder creation on open

* Fixed List implementation. Also lots of debug logging.

* Removed debug code

* Added a scope guard for WritePlaceHolder

* Manually prevent placeholder/content appending

* Revert "Removed debug code"

This reverts commit d6ff261fcc.

* Always cache placeholder file. Switch to ftell for preventing appending

* Universally use EnsureEnabled

* Abstract away file writing logic

* Misc cleanup

* Refactor placeholder cacheing

* Remove debug code (again)

* Revert "Remove debug code (again)"

This reverts commit 168447d80e.

* Misc changes

* Fixed file modes

* Fixed ContentId/PlaceHolderId alignment

* Improved type safety

* Fixed reinitialization

* Fixed doubleup on path creation

* Remove debug code

* Fixed 1.0.0 booting

* Correct amount of add on content

* Correct main thread stack size

* lr: Introducing registered data

* Reorder stratosphere Makefile

* Move results to libstrat

* lr: Cleanup lr_redirection

* lr: lr_manager tweaks

* lr: Imrpoved path handling and adjust ResolveAddOnContentPath order

* lr: Organise types

* Add eof newlines

* lr: Eliminate unnecessary vars

* lr: Unnecessary vars 2 electric boogaloo

* lr: Various helpers

* lr: RegisteredLocationResolver helpers

* ncm: Move ncm_types to libstrat

* ncm: Misc cleanup

* Updated AddOnContentLocationResolver and RegisteredLocationResolver to 9.0.0

* Finished updating lr to 9.0.0

* Updated NCM to 9.0.0

* Fix libstrat includes

* Fixed application launching

* title_id_2 -> owner_tid

* Updated to new-ipc

* Change to using pure virtuals

* Title Id -> Program Id

* Fixed compilation against master

* std::scoped_lock<> -> std::scoped_lock

* Adopted R_UNLESS and R_CONVERT

* Prefix namespace to Results

* Adopt std::numeric_limits

* Fixed incorrect error handling in ReadFile

* Adopted AMS_ABORT_UNLESS

* Adopt util::GenerateUuid()

* Syntax improvements

* ncm_types: Address review

* Address more review comments

* Updated copyrights

* Address more feedback

* More feedback addressed

* More changes

* Move dispatch tables out of interface files

* Addressed remaining comments

* lr: move into libstratosphere

* ncm: Fix logic inversion

* lr: Add comments

* lr: Remove whitespace

* ncm: Start addressing feedback

* ncm: Cleanup InitializeContentManager

* lr: support client-side usage

* lr_service -> lr_api

* ncm: Begin refactoring content manager

* ncm: More content manager improvements

* ncm: Content manager mount improvements

* ldr: use lr bindings

* lr bindings usage: minor fixes

* ncm/lr: Pointer placement

* ncm: placeholder accessor cleanup

* ncm: minor fixes

* ncm: refactor rights cache

* ncm: content meta database cleanup

* ncm: move content meta database impl out of interface file

* ncm: Use const ContentMetaKey &

* ncm: fix other non-const ContentMetaKey references

* ncm: content meta database cleanup

* ncm: content storage fixes for 2.0.0

* ncm: add missing end of file newlines

* ncm: implement ContentMetaReader

* ncm: client-side api

* ncm: trim trailing spaces

* ncm: FS_MAX_PATH-1 -> fs::EntryNameLengthMax

* ncm: Use PathString and Path

* fs: implement accessor wrappers for ncm

* fs: implement user fs wrappers

* fs: add MountSdCard

* ncm: move to content manager impl

* ncm: fix up main

* kvdb: use fs::

* fs: Add wrappers needed for ncm

* ncm: use fs bindings, other refactoring

* ncm: minor fixes

* fsa: fix ReadFile without size output

* fs: add substorage, rom path tool

* ncm: fix dangling fsdev usage

* fs: fix bug in Commit

* fs: fixed incorrect mode check

* fs: implement Mount(System)Data

* ncm: don't delete hos

* results: add R_SUCCEED_IF

* ams-except-ncm: use R_SUCCEED_IF

* ncm: added comments

* ncm: fix api definitions

* ncm: use R_SUCCEED_IF

* pm: think of the savings

* ncm: employ kernel strats

* ncm: Nintendo has 5 MiB of heap. Give ourselves 4 to be safe, pending analysis

* ncm: refactor IDs, split types header into many headers

* ams.mitm: use fs bindings instead of stdio

* fs: SystemData uses SystemDataId

* ncm: improve meta-db accuracy

* ncm: inline getlatestkey

* fs: improve UnsupportedOperation results

* fs: modernize mount utils

* ams: misc fixes for merge-errors

* fs: improve unsupportedoperation results

* git subrepo pull emummc

subrepo:
  subdir:   "emummc"
  merged:   "d12dd546"
upstream:
  origin:   "https://github.com/m4xw/emuMMC"
  branch:   "develop"
  commit:   "d12dd546"
git-subrepo:
  version:  "0.4.1"
  origin:   "???"
  commit:   "???"

* util: add boundedmap

* ncm: minor style fixes

* ncm: don't unmount if mounting fails

* lr: bug fixes

* ncm: implement ncm.for-initialize + ncm.for-safemode

* lr: ncm::ProgramId::Invalid -> ncm::InvalidProgramId

* ncm: fix open directory mode on 1.0.0

* ncm: fix fs use, implement more of < 4.0.0 for-initialize/safemode

* ncm: implement packagedcontent -> content for building metadb

* ncm: fix save data flag management

* ncm: address some review suggestions (thanks @leoetlino!)

* updater: use fs bindings

* fs: implement MountCode

* fs: prefer make_unique to operator new

* ncm: implement remaining ContentMetaDatabaseBuilder functionality

Co-authored-by: Michael Scire <SciresM@gmail.com>
 2020-03-08 00:06:23 -08:00
Michael Scire
dc1404061c fs.mitm: implement layered html redirection (closes #814) 2020-02-25 16:44:36 -08:00
Michael Scire
8da705d40b fs.mitm: support html redirection (non-layered, for #814) 2020-02-25 06:41:53 -08:00
Michael Scire
40400aee1f ams: revamp assertion system 2020-02-22 23:05:14 -08:00
Michael Scire
6ecf04c3b7 find -exec sed -i'' -e 's/2018-2019 Atmo/2018-2020 Atmo/g' {} + 2020-01-24 02:10:40 -08:00
Michael Scire
3e2ec256ab fs.mitm: mitm sdb (allows common sysarchive redir) 2019-12-31 01:57:26 -08:00
Michael Scire
5a15d62b8b fs.mitm: require less contiguous memory to build romfs 2019-12-30 05:02:07 -08:00
Michael Scire
d0404f3cc9 fs.mitm: lazily initialize sd-romfs metadata 2019-12-30 03:23:40 -08:00
Michael Scire
2ae298de24 fs.mitm: conserve memory when building romfs paths 2019-12-30 02:51:32 -08:00
Michael Scire
06e4b94674 fs.mitm: only iter romfs dir if it exists (closes #763) 2019-12-16 17:31:53 -08:00