This commit marks a significant architectural change, moving away from
the Docker-OSX VM based approach to a macOS installer creation method
leveraging `corpnewt/gibMacOS` for downloading official macOS assets.
Key changes in this submission:
1. **Core Workflow Change (`main_app.py`):**
* I removed Docker-dependent UI and associated logic.
* I introduced new logic to manage downloading macOS installer assets
using `gibMacOS.py` (assumed to be present in a `./scripts` subdirectory or PATH).
* The UI flow is now:
* Step 1: Download macOS Installer Assets.
* Step 2: Create Bootable USB Installer (using platform-specific writers).
* The USB writing process now expects `macos_download_path` (where
`gibMacOS` saves files) instead of qcow2 image paths.
* Application version in "About" dialog updated to reflect major change.
2. **Refactored `usb_writer_linux.py` for Installer Creation:**
* `__init__` updated to accept `macos_download_path`.
* The `format_and_write` method was substantially rewritten to:
* Partition USB (EFI FAT32, Main HFS+).
* Implement `_find_gibmacos_asset` and `_get_gibmacos_product_folder`
to locate key files (BaseSystem.dmg, InstallAssistant.pkg, etc.)
within the `gibMacOS` download structure.
* Implement `_extract_hfs_from_dmg_or_pkg` to extract the
bootable BaseSystem HFS image from downloaded DMGs/PKGs.
* Write the extracted BaseSystem HFS image to the main USB HFS+
partition.
* Copy essential installer support files (`BaseSystem.dmg`,
`BaseSystem.chunklist`, `InstallInfo.plist`, and the main installer
PKG like `InstallAssistant.pkg`) to standard locations on the USB's
HFS+ partition to make it a more complete installer.
* Set up an OpenCore EFI on the USB's ESP by copying from a
`EFI_template_installer` directory and conditionally calling
`plist_modifier.enhance_config_plist` on its `config.plist`.
* Updated dependency checks (added `7z`, `sgdisk`; removed `qemu-img`, `kpartx`
as direct dependencies for this script's new role).
3. **Created `EFI_template_installer` Directory:**
* Established a basic directory structure for a minimal OpenCore EFI
(EFI/BOOT/BOOTx64.efi, EFI/OC/*, config-template.plist, and placeholder
files for common drivers, kexts, ACPI). This template is used by
`usb_writer_linux.py`.
**Note:**
* `usb_writer_macos.py` and `usb_writer_windows.py` have **not yet** been
refactored for this new installer-based workflow and will require
significant updates in subsequent steps.
* The logic for copying *all* necessary files to make a fully complete
macOS installer partition in `usb_writer_linux.py` (beyond BaseSystem
and key packages) is still a work-in-progress and will need further
refinement based on macOS installer structure research.
This commit lays the foundation for the new installer-centric approach as per
your recent feedback.
This commit introduces several major enhancements:
1. **Experimental `config.plist` Auto-Enhancement (Linux Host for Detection):**
* `linux_hardware_info.py`: Added audio codec detection.
* `plist_modifier.py`:
* Uses detected audio codecs for more accurate `layout-id` selection.
* Expanded mappings for Intel Alder Lake iGPUs, more audio devices, and Ethernet kexts.
* Refined NVIDIA GTX 970 `boot-args` logic based on target macOS version and iGPU presence.
* Creates a `.backup` of `config.plist` before modification and attempts restore on save failure.
* Integrated into `main_app.py` with a user-selectable experimental checkbox.
2. **Reworked `README.md`:**
* Completely rewritten for "Skyscope" branding and project vision.
* Details all current features, including platform-specific USB writing (manual Windows dd step).
* Comprehensive prerequisites, including `apfs-fuse` build dependencies for Debian.
* Updated usage guide and future enhancement plans. Version set to 0.8.2.
3. **UI/UX Enhancements for Task Progress:**
* Added a QTimer-driven text-based spinner to the status bar for active operations.
* Centralized UI state management (`_set_ui_busy`, `update_all_button_states`) for consistent feedback and control enabling/disabling.
* Refactored completion/error handling into generic slots.
4. **Improved Windows USB Writing Guidance:**
* `usb_writer_windows.py` now uses `diskpart` to fetch and display the
macOS partition number and byte offset, providing more specific details
for your manual `dd` operation.
5. **Debian 13 "Trixie" Compatibility:**
* Reviewed dependencies and updated `README.md` with specific notes for
`hfsprogs` and `apfs-fuse` installation on Debian-based systems.
This set of changes makes the application more intelligent in its OpenCore
configuration attempts, improves your feedback during operations, and provides
much more comprehensive documentation, while also advancing the capabilities
of the platform-specific USB writers.
This commit builds upon the previous cross-platform USB writer implementation
by adding an experimental feature to auto-enhance the OpenCore `config.plist`
and reviewing Debian 13 compatibility for dependencies.
Key new features and changes in this iteration:
1. **Experimental `config.plist` Auto-Enhancement:**
* Introduced `linux_hardware_info.py` to gather CPU details and PCI device
information (GPU, Audio, Ethernet) on Linux systems.
* Created `plist_modifier.py` with logic to:
* Load an existing `config.plist`.
* Apply targeted modifications based on detected Linux host hardware:
* Inject Intel iGPU `DeviceProperties` (AAPL,ig-platform-id, device-id).
* Set audio `layout-id` for common audio controllers.
* Ensure relevant Ethernet kexts are enabled.
* Implement specific boot-arg adjustments for NVIDIA GTX 970 based on the
target macOS version and presence of an iGPU (e.g., `nvda_drv=1` for
High Sierra, `nv_disable=1` for newer macOS if iGPU is primary).
* Create a backup (`config.plist.backup`) before modifying the plist.
* Integrated this feature into `main_app.py` with a UI checkbox (default off)
to enable it during the USB writing process (currently, hardware detection
for enhancement is Linux-only).
* The USB writer modules (`usb_writer_linux.py`, `usb_writer_macos.py`) now
conditionally call the plist enhancement logic.
2. **Debian 13 "Trixie" Compatibility Review:**
* Reviewed dependencies for Linux USB writing (`qemu-img`, `parted`, `kpartx`,
`rsync`, `mkfs.vfat`, `mkfs.hfsplus`, `apfs-fuse`, etc.) for expected
availability on Debian 13.
* Updated `README.md` with more specific guidance for Debian users on
installing `hfsprogs` and `apfs-fuse` (including typical build
dependencies for compiling `apfs-fuse` from source).
3. **Documentation Updates (`README.md`):**
* Updated to version 0.8.1 (Alpha).
* Added details about the new experimental `config.plist` enhancement feature,
its current limitations (Linux-only hardware detection), and user guidance.
* Incorporated notes from the Debian 13 compatibility review.
4. **Consolidated Previous Work:**
* This submission includes all prior work from this session, such as the
cross-platform USB writer shells (Linux, macOS, Windows PoC), UI/UX
enhancements (status bar, progress bar), improved Docker interaction
(explicit `docker pull`), and privilege checks.
The application now offers a more advanced, albeit experimental, capability to
tailor the OpenCore configuration, alongside its core functionality of
automating Docker-OSX VM setup and bootable USB creation.
This major update brings several key features and improvements:
1. **Cross-Platform USB Writing:**
* **Linux:** I refactored USB writing (`usb_writer_linux.py`) to use file-level
copy (`rsync`) for the main macOS partition, correctly handling various
USB sizes and dependencies like `apfs-fuse`.
* **macOS:** I implemented USB writing (`usb_writer_macos.py`) using native
tools (`diskutil`, `hdiutil`, `rsync`) for a fully automated file-level
copy process for both EFI and macOS partitions.
* **Windows:** I added initial USB writing support (`usb_writer_windows.py`)
automating EFI partition setup and file copy (using `diskpart`, `7z.exe`,
`robocopy`). Writing the main macOS system image currently requires a
guided manual step using an external 'dd for Windows' utility.
2. **Enhanced Docker Interaction:**
* I added an explicit `docker pull` step before `docker run`, with progress
streamed to the GUI, ensuring the image is present and up-to-date.
3. **Improved Privilege Handling & USB Detection:**
* I implemented checks for admin/root privileges before initiating USB writing
operations on all platforms.
* I significantly improved USB drive detection on Windows by using PowerShell/WMI
to query physical USB disks, populating a selectable dropdown for you.
Manual disk ID input is now a fallback.
4. **UI/UX Enhancements:**
* I added an indeterminate `QProgressBar` and integrated `QMainWindow.statusBar()`
messages to provide better visual feedback during long operations.
* I centralized UI state management (`_set_ui_busy` method) for more
consistent enabling/disabling of controls.
* I refactored how I handle completion and errors for cleaner UI updates.
5. **Documentation:**
* I updated `README.md` to reflect new features, platform-specific
prerequisites (including `hfsprogs`, `apfs-fuse` for Linux, `7z.exe`
for Windows), usage instructions, known limitations, and current version.
6. **Code Structure:**
* I introduced `usb_writer_macos.py` and `usb_writer_windows.py`.
* I updated `main_app.py` extensively to integrate these features and manage
the enhanced workflow.
This version represents a significant step towards a fully cross-platform and
more user-friendly application, addressing key feedback on USB writing
reliability and user guidance.
This commit introduces USB writing capabilities for Linux, macOS, and an initial
proof-of-concept for Windows. It also includes the preceding work on GUI development,
Docker orchestration, and image extraction.
Key features and changes:
1. **GUI (PyQt6):**
* I've created a user interface for macOS version selection, Docker interaction, image
extraction, USB drive selection, and operational feedback.
* I've also made platform-specific UI adjustments for USB drive identification on Windows
(manual disk number input).
2. **Docker Orchestration:**
* I've set up the system to build and run Docker-OSX commands in a separate thread.
* It now streams Docker logs to the GUI.
* It also persists containers with unique names for file access.
* And finally, it manages container lifecycle (stop/remove).
3. **Image Extraction:**
* This allows you to copy `mac_hdd_ng.img` (main OS) and `OpenCore.qcow2` (EFI)
from the Docker container to the local filesystem.
4. **USB Writing - Linux (`usb_writer_linux.py`):**
* I've refactored this to use file-level copy (`rsync`) for the main macOS system
partition, correctly handling various USB sizes.
* It uses `qemu-img`, `parted`, `kpartx`, `apfs-fuse`, `mkfs.hfsplus`, `rsync`.
* I've also automated partitioning, formatting, EFI copy, and macOS system copy.
5. **USB Writing - macOS (`usb_writer_macos.py`):**
* I've implemented this using `diskutil`, `hdiutil`, `qemu-img`, and `rsync`.
* I've also automated partitioning, formatting, EFI copy (via raw image attach and
rsync), and macOS system copy (via raw image attach and rsync).
6. **USB Writing - Windows (`usb_writer_windows.py`):**
* This is an initial implementation using `diskpart` for partitioning and EFI setup.
* It uses `qemu-img` for image conversion.
* It relies on `7z.exe` (external) for EFI file extraction from raw image and
`robocopy` for copying EFI files to USB.
* **Main macOS system image writing is currently a guided manual step**,
instructing you to use an external 'dd for Windows' utility. This
is due to the complexity of automating raw HFS+/APFS partition writing
safely on Windows without commercial filesystem drivers.
7. **Cross-Platform Integration (`main_app.py`):**
* I now dispatch to the appropriate platform-specific
writer module (Linux, macOS, Windows).
* GUI elements adapt for platform-specific needs (e.g., Windows disk ID input).
* All long-running operations are threaded with progress updates.
8. **Documentation:**
* I've created a `README.md` file, covering overview, features, prerequisites (including
OS-specific tools), usage instructions, known limitations (especially for
Windows USB writing and the macOS image sizing for `dd`), and planned
enhancements.
9. **General Structure:**
* I've organized the code into `main_app.py`, `utils.py`, `constants.py`,
`usb_writer_linux.py`, `usb_writer_macos.py`, and `usb_writer_windows.py`.
This work consolidates all development up to providing initial USB
creation capabilities across the three target platforms, with Linux and macOS
being mostly automated and Windows having a defined path with some manual
intervention required for the main system write.
Implements the "Skyscope macOS on PC USB Creator Tool" with core features:
1. **GUI (PyQt6):** For macOS version selection, Docker interaction, image extraction, USB selection, and feedback.
2. **Docker Orchestration:**
* Builds and runs Docker-OSX commands.
* Streams logs to GUI; runs Docker in a separate thread.
* Persists containers with unique names for file access.
3. **Image Extraction:**
* Copies `mac_hdd_ng.img` and `OpenCore.qcow2` from the container.
* Manages container lifecycle (stop/remove).
4. **USB Writing (Linux PoC):**
* `usb_writer_linux.py` handles partitioning, formatting, and image writing.
* Uses `qemu-img`, `parted`, `kpartx`, `rsync`, `mkfs.vfat`, `dd`.
* Includes data erasure warning.
* Known issue: `dd` for main image writes full sparse size; needs change to file-level copy.
5. **iGPU Config Review:** Confirmed Docker-OSX's use of WhateverGreen.
6. **Documentation:** Added `README.md` with overview, prerequisites, usage, known issues, and future work based on your feedback (image sizing, explicit pull, privilege handling).
The application files include `main_app.py`, `utils.py`, `constants.py`, `usb_writer_linux.py`, and `README.md`.
This commit reflects the state after incorporating your feedback regarding visibility of all created files in the repository.
I've implemented the initial GUI application "Skyscope macOS on PC USB Creator Tool" using PyQt6.
Key features I've implemented:
1. **macOS Version Selection:** You can select the desired macOS version.
2. **Docker Orchestration:**
* I will build and display `docker run` commands for Docker-OSX.
* I will run Docker containers in a separate thread with output streamed to the GUI.
* I will persist containers (no --rm) and use unique names for later access.
3. **Image Extraction:**
* You can copy `mac_hdd_ng.img` and `OpenCore.qcow2` from the
container to your local filesystem after macOS installation in QEMU.
* I provide options to stop and remove the container.
4. **USB Drive Management (Linux PoC):**
* I will detect potential USB drives using `psutil`.
* I've implemented USB writing for Linux systems (`usb_writer_linux.py`):
* I will check for dependencies (qemu-img, parted, kpartx, rsync, mkfs.vfat).
* I will partition the USB drive (GPT, EFI, macOS HFS+).
* I will convert QCOW2 images (OpenCore and macOS base) to raw.
* I will copy EFI files to the ESP using kpartx and rsync.
* I will write the main macOS image using `dd` (Note: known issue - this currently
writes the full sparse size; I need to change this to file-level copy).
* I will include a data erasure warning before writing to USB.
5. **iGPU Configuration Review:**
* I've confirmed that the default OpenCore config from Docker-OSX includes
WhateverGreen for generic Intel iGPU support.
The application structure includes `main_app.py`, `utils.py`, `constants.py`,
and `usb_writer_linux.py`. I manage background tasks in QThreads to keep
the GUI responsive.
Further work based on your feedback will address:
- Modifying USB writing to use file-level copies for the main macOS system
to correctly handle disk sizes.
- Adding an explicit `docker pull` stage.
- Improving privilege checks for USB operations.
When doing `docker build`, docker (tested with 27.0.3) prints a warning:
LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 238)
This commit fixes it by using proper syntax.
Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
