[buildiso] profiles need to be self-contained and not rely on system specific confs to generate identical images
Created by: Sleaker
Right now the manjaro-tools.conf interacts in a sub-optimal way with generating the ISO via buildiso. If I download a git version of isoprofiles I can not be certain that I will be generating an identical ISO as someone else that has released an ISO due to possible differences in the manjaro-tools.conf, either on their system or in their .config. ISO profiles should inherently define ALL aspects of the CD at build time to prevent discrepancies with user-configs.
In addition to the possibility of differences between user configurations, I should not need to change a .config back and forth between different settings to change per-ISO values for package versions (ex stable vs unstable)
Expected: iso-profiles define all aspects of their build including package versioning.
Current: manjaro-tools.conf specifies versioning for kernel.
This is a regression of functionality that was present in manjaroiso.
- Thanks
Activity
Created by: aadityabagga
options.conf in each profile would be useful in my opinion, because if someone builds multiple profiles one has to edit one's manjaro-tools.conf before building to suit each profile...
Also as Sleaker mentioned it leads to issue that a profile is incomplete without the config.
A possible solution could be that an options.conf in a profile could be added by user which would override the global settings in manjaro-tools.conf, or vice-versa.
-
Created by: udeved
I centralized settings for the moment. Once cli-instaler will be absolutely deprecated, we can switch it over to a profile.conf. However, some stuff will remain likely centralized, such as kernel version for multi iso build of a given release series. Its more like centralizing per profile initsys, displaymanager, plymouth theme and maybe a DM theme in one file.
-
Created by: Sleaker
@udeved - keeping kernel versions centralized outside of the profile is unacceptable for me from a design point of view as it means that multiple developers working on a project can not be certain that they will generate the same ISO. If this is the case then I may as well build my own set of scripts that generates an ISO seperate from manjaro. I would think that the whole point of having templates/profiles to begin with is to be able to gaurantee that I can generate the same ISO as someone else without needing to download some user-specific configuration that can't actually be overwritten from the profile itself.
Maybe that's not the intention of manjaro-tools, but as I mentioned above, this is a clear regression of current manjaroiso functionality as the scripts were self-contained previously.
-
Created by: udeved
I don't get your point. The chance that a standardized iso will be generated increases with a centralized config and its apparently the same mechanism as with manjaroiso, just that the location of the file changed, and code is separated from the profiles.
If this is totally unacceptable to you, I suggest you indeed start your own scripts then, because manjaro-tools are not manjaroiso.
-
Created by: Sleaker
@udeved - I'm trying to say that the centralized config is not contained inside of the profile distribution and this is a poor design decision when it comes to collaboration. Seeing as manjaro-tools is a direct replacement for manjaroiso I would think that maintaining much of the functionality is wanted, especially one that directly affects the creation of the iso such as versions of packages. I have a hard time understanding why you wouldn't want to be able to simply send someone a copy of the profile and have them be able to generate the whole image without them needing to setup an identical manjaro-tools.conf. Is there a possibility of allowing the profiles to have a manjaro-tools.conf themselves?
-
Created by: udeved
Functionality is added. Please understand that manjaro-tools do not target end users, they are development tools.
Centralized config ensures you build same isos with same kernels for one release series, eg using a buildset. If I use buildsets, I do not want to fiddle with each profile I want to batch build.
I don't understand why you wouldn't send in your example simply a manjaro-tools.conf
The reason for centrailzed config are:
a) cli-installer(manjaroiso and cli-installer had same code base which is slowly being reduced without brealking cli) b) buildsets c) at current point of development, I simply don't want individual profile settings because of a)
-
Created by: Sleaker
@udeved - lets say I want to create an openrc image, then create a non-openrc image. I want to create both of these images for kernel318 and kernel319. Can I currently accomplish this on a per-profile basis or do I need to edit the manjaro-tools.conf for each different ISO image?
Currently it's my understanding that I would need to edit the manjaro-tools.conf to generate each image properly.
I'm attempting to help on development so please understand this isn't necessarily about 'targeting users.' I'm trying to collaborate on a project and simply bringing up my use-case and the hurdles I've already had to deal with to get setup.
Can you explain what you meant by 'functionality is added?' I don't see a reference to what functionality you were stating was added, or if this meant that a portion of what I'm asking for is already in there (and maybe just not documented?)
Also thanks much for taking the time to discuss this, I really appreciate it!
-
Created by: udeved
You would create a buildset per init.
And yes, you got to edit the kernel version per build run. As said, the concept is geared towards buildsets, ie a single profile build is just one case to consider.
The point is, it is now possible to build say 10 systemd profiles in one buildiso call for eg version manjaro-0.9.0, and having a centralized config helps a lot with this approach because all 0.9.0 versions use kernel >=319.. I do see the need to configure eventually plymouth per profile.
-
Created by: Sleaker
@udeved - the Packages are symlinked together for other profiles like this.. Why wouldn't that also be an option? Or allow an override to be specified in the profile?
If I created a PR for this functionality would it simply be rejected or is this just something you don't have the manpower to focus on at this time?
-
Created by: Sleaker
Accidentally hit close there. I understand the idea behind central conf because most (not all) images use similar setups for kernel/init. But my thought with symlinks was that they are currently used for base Packages for instance in the xfce profiles etc. If you want to create a central conf can't this be replicated for this use? The central conf isn't good especially for batching when I want to batch run 2 different init systems on ISOs.
I understand you may not want to put effort in here. So my question now comes down to, if I do the work for allowing either a command line switch to allow local manjaro-tools.conf to be used when running buildiso, or a per-profile file to override if present would this be accepted? And if so which one is preferable?
-
Created by: udeved
I am very reluctant to do services per profile. The point of having buildsets is to standardize a release series, or else don't use a buildset.
@Sleaker
You don't understand the buildsets, you need two buildsets for what you want. You want to build 2 openrc and 4 systemd profiles. You create one openrc buiildset, and one systemd buildset and call the buildset insted of the profile.
Naturally, chroots are not identical for different inits.
-
Created by: udeved
@Sleaker
See here the sample buildsets installed. https://github.com/manjaro/manjaro-tools/tree/devel/sets/iso
As said, once cli-installer is deprecated, some stuff may move to profile. It is simply a matter of compromise between having a standardized release series, and allowing profile based configs. Individual config normally cause more trouble and are eventual additional sources of error.
-
Created by: philmmanjaro
@Sleaker : which parts do you want to change? Kernel? Maybe we add a override so you can build still in batch but with different kernels. From the release point of view a simply batch script might do it also for now:
buildiso -p set1 sed -i -e 's|manjaro_kernel="linux319"|manjaro_kernel="linux40"|g' /path/to/manjaro-tools.conf buildiso -p set2
-
Created by: aadityabagga
Maybe something like the following could be supported.. https://github.com/aadityabagga/manjaro-tools-iso-profiles/blob/master/xfce-openbox-openrc/services
-
Created by: udeved
@philmmanjaro
Should we merge the devel branch of profiles in master? I intend to probably break devel, since I had for long time the plan to introduce a profile.conf containing initsys and DM values. Technically, I believe I decoupled cli from manjaro-tools(and former options), so manjaro-tools.conf is not a requirement for cli installer anymore. But I have to make sure of that.
-
Created by: Kirek
Well I think his point is not building two isos with different kernels, but sharing profiles with other developers and both of them make the exact same profile.
Let's say two developers are working on two profiles, but each profile have different settings for manjaro-tools.conf (doesn't matter why, or which ones are, that's not the point, they are just different).
To share this profiles they would need to send the profiles and the two manjaro-tools.conf, build one profile with its manjaro-tools.conf, then change again the manjaro-tools.conf and build the other one. Also you would have to backup the original manjaro-tools.conf...
So having the buildiso contained within the profiles make sense. Imo this could be solved adding a profile.conf in shared, if the profiles doesn't have a profile.conf, default to the shared one.
-
Created by: Sleaker
@Kirek - that was my initial idea which seems to have been lost, thanks for re-iterating that point.
-
Created by: udeved
The problem is the cli installer guys. I has a reason I divided the majaro-tools.conf in sections, so to know what cli needs etc. I am relieved we dropped 0.8 cli. We are not at the end of devel, its just a lot of plan and stuff only in my head, not on paper. ;) But before I would introduce any profile config, I'd like to get the logger working before that. So next focus is logger.
-
-
Created by: philmmanjaro
profile.conf seems fine to me. Another point is, we have to think about branding. sonarlinux is using Manjaro as base and they need also some features. So any manjaro branding should be not hardcoded. Calamares uses also a branding option. Sure we might think it is not needed, but more and more people will use our distro as their base and might want to change the branding.
@udeved: If you like to add new features, please use a branch as we do in Calamares. Work on it as long as you need to and start a merge request. Then we can discuss the topic much easier. And always, if you release a stable we have to merge also the working profiles snapshot over to master branch and create a tarball.
-
Created by: udeved
I'll close this, since feature has been implemented. If errors occur, please open a new issue.
@philmmanjaro
If the unstable changes have been tested a bit more, I'd do a 0.9.7 release with these changes and updates livecd and profiles. So 0.9.7.x will be bug fix releases. Result is, we got profile config a milestone earlier than originally planned, I was gonna make this transition slowly, now we got it in one step.