pamac issueshttps://gitlab.manjaro.org/applications/pamac/-/issues2022-02-21T11:58:05Zhttps://gitlab.manjaro.org/applications/pamac/-/issues/1237AUR Archive Diffs (WIP)2022-02-21T11:58:05ZKevin MorrisAUR Archive Diffs (WIP)Hello pamac,
AUR developer here. We are running into an issue again with the amount of traffic used to maintain pamac's usage of the AUR, and this time its the result of archive transfers.
We have had this feature planned for a while n...Hello pamac,
AUR developer here. We are running into an issue again with the amount of traffic used to maintain pamac's usage of the AUR, and this time its the result of archive transfers.
We have had this feature planned for a while now, and we're going to hop on it now and get it done. Basically, this feature will end up storing snapshots of archives, and when presented with a `Last-Modified` from the user, we will deduce we should serve out either the entire archive (if your `Last-Modified` is too old), or a particular diff that your `Last-Modified` fits into. We will provide sample code so that these diffs can be easily consumed and used in your applications.
Doing this should pretty much kill the over-consumption of resources on the AUR, as long as people's machines reliably stay up to date.
We'll keep you updated here regarding this. I'm trying to solve this as quick as possible, but this feature should be ironed out so that it's stable for a long time to come, so we're not going to be rushing it.
We also have some new .sha256 checksums which should probably be coupled with the next patchset regarding these archives, so that you can verify integrity of each download; we will describe this when the diffs are ready.
Best,
Kevinhttps://gitlab.manjaro.org/applications/pamac/-/issues/1223`Refreshing AUR` stage downloads a 9.3 MiB file and eats about 1.5 GiB of RAM...2022-02-20T09:11:48Zlivemuziek`Refreshing AUR` stage downloads a 9.3 MiB file and eats about 1.5 GiB of RAM memory## Pamac --version
Pamac 10.3.0-2 - libpamac 11.2.0
### Variant in use
CLI
## Distribution
Manjaro (unstable branch of updates)
## Desktop environment
KDE
## What's not working
`pamac` consumes 1.5 GB of RAM during the `Refreshing ...## Pamac --version
Pamac 10.3.0-2 - libpamac 11.2.0
### Variant in use
CLI
## Distribution
Manjaro (unstable branch of updates)
## Desktop environment
KDE
## What's not working
`pamac` consumes 1.5 GB of RAM during the `Refreshing AUR` stage.
## How to reproduce?
constantly: `pamac update --force-refresh`
from time to time: `pamac update` or `pamac install any_package_name`
## More information (optional)
command called and it's output:
```text
pamac update --force-refresh
Preparing...
Synchronizing package databases...
Refreshing core.db...
Refreshing extra.db...
Refreshing community.db...
Refreshing multilib.db...
Refreshing chaotic-aur.db...
Refreshing core.files...
Refreshing extra.files...
Refreshing community.files...
Refreshing multilib.files...
Refreshing chaotic-aur.files...
Refreshing AUR...
Nothing to do.
Transaction successfully finished.
```
A few days ago I noted that new `Refreshing AUR...` stage appeared while calling force refresh.
<details><summary>I have semi-garbage PC</summary>
```text
~/Desktop ❯ inxi -Fazy1
System:
Kernel: 5.16.2-1-MANJARO x86_64
bits: 64
compiler: gcc
v: 11.1.0
parameters: BOOT_IMAGE=/@/boot/vmlinuz-5.16-x86_64 root=UUID=b3cf0f6a-55e1-4e25-83d5-88c0781c30e3 rw rootflags=subvol=@ quiet cryptdevice=UUID=ef44cf91-5209-4ff4-95bf-ffb124a71d7e:luks-ef44cf91-5209-4ff4-95bf-ffb124a71d7e root=/dev/mapper/luks-ef44cf91-5209-4ff4-95bf-ffb124a71d7e apparmor=1 security=apparmor udev.log_priority=3 ipv6.disable=1
Desktop: KDE Plasma 5.23.5
tk: Qt 5.15.2
wm: kwin_x11
vt: 1
dm: SDDM
Distro: Manjaro Linux
base: Arch Linux
Machine:
Type: Laptop
System: ASUSTeK
product: K50IJ
v: 1.0
serial: <superuser required>
Mobo: ASUSTeK
model: K50IJ
v: 1.0
serial: <superuser required>
BIOS: American Megatrends
v: 218
date: 04/09/2010
Battery:
Device-1: hidpp_battery_0
model: Logitech Wireless Mouse
serial: <filter>
charge: 55% (should be ignored)
rechargeable: yes
status: Discharging
Device-2: hidpp_battery_1
model: Logitech Wireless Keyboard K360
serial: <filter>
charge: 100% (should be ignored)
rechargeable: yes
status: Discharging
CPU:
Info:
model: Celeron T3100
bits: 64
type: MCP
arch: Core Penryn
family: 6
model-id: 0x17 (23)
stepping: 0xA (10)
microcode: 0xA0B
Topology:
cpus: 1
cores: 2
smt: <unsupported>
cache:
L1: 128 KiB
desc: d-2x32 KiB; i-2x32 KiB
L2: 1024 KiB
desc: 1x1024 KiB
Speed (MHz):
avg: 1895
min/max: N/A
cores:
1: 1895
2: 1895
bogomips: 7584
Flags: ht lm nx pae sse sse2 sse3 ssse3
Vulnerabilities:
Type: itlb_multihit
status: KVM: VMX unsupported
Type: l1tf
mitigation: PTE Inversion
Type: mds
status: Vulnerable: Clear CPU buffers attempted, no microcode; SMT disabled
Type: meltdown
mitigation: PTI
Type: spec_store_bypass
status: Vulnerable
Type: spectre_v1
mitigation: usercopy/swapgs barriers and __user pointer sanitization
Type: spectre_v2
mitigation: Full generic retpoline, STIBP: disabled, RSB filling
Type: srbds
status: Not affected
Type: tsx_async_abort
status: Not affected
Graphics:
Device-1: Intel Mobile 4 Series Integrated Graphics
vendor: ASUSTeK
driver: i915
v: kernel
bus-ID: 00:02.0
chip-ID: 8086:2a42
class-ID: 0300
Device-2: Chicony 2.0M UVC Webcam / CNF7129
type: USB
driver: uvcvideo
bus-ID: 1-3:2
chip-ID: 04f2:b071
class-ID: 0e02
serial: <filter>
Display: x11
server: X.org 1.21.1.3
compositor: kwin_x11
driver:
loaded: modesetting
alternate: fbdev,vesa
resolution: <missing: xdpyinfo>
Message: Unable to show advanced data. Required tool glxinfo missing.
Audio:
Device-1: Intel 82801I HD Audio
vendor: Santa Cruz Operation
driver: snd_hda_intel
v: kernel
bus-ID: 00:1b.0
chip-ID: 8086:293e
class-ID: 0403
Sound Server-1: ALSA
v: k5.16.2-1-MANJARO
running: yes
Sound Server-2: JACK
v: 1.9.20
running: no
Sound Server-3: PulseAudio
v: 15.0
running: yes
Sound Server-4: PipeWire
v: 0.3.43
running: yes
Network:
Device-1: Qualcomm Atheros AR8121/AR8113/AR8114 Gigabit or Fast Ethernet
vendor: ASUSTeK
driver: ATL1E
v: N/A
modules: atl1e
port: ec00
bus-ID: 02:00.0
chip-ID: 1969:1026
class-ID: 0200
IF: enp2s0
state: up
speed: 100 Mbps
duplex: full
mac: <filter>
Bluetooth:
Device-1: Realtek Bluetooth Radio
type: USB
driver: btusb
v: 0.8
bus-ID: 6-1:2
chip-ID: 0bda:8771
class-ID: e001
serial: <filter>
Report: rfkill
ID: hci0
rfk-id: 2
state: up
address: see --recommends
Drives:
Local Storage:
total: 232.89 GiB
used: 90.71 GiB (38.9%)
SMART Message: Unable to run smartctl. Root privileges required.
ID-1: /dev/sda
maj-min: 8:0
vendor: Seagate
model: ST9250315AS
size: 232.89 GiB
block-size:
physical: 512 B
logical: 512 B
speed: 3.0 Gb/s
type: HDD
rpm: 5400
serial: <filter>
rev: SDM1
scheme: MBR
Partition:
ID-1: /
raw-size: 232.88 GiB
size: 232.88 GiB (100.00%)
used: 90.71 GiB (39.0%)
fs: btrfs
dev: /dev/dm-0
maj-min: 254:0
mapped: luks-ef44cf91-5209-4ff4-95bf-ffb124a71d7e
ID-2: /home
raw-size: 232.88 GiB
size: 232.88 GiB (100.00%)
used: 90.71 GiB (39.0%)
fs: btrfs
dev: /dev/dm-0
maj-min: 254:0
mapped: luks-ef44cf91-5209-4ff4-95bf-ffb124a71d7e
ID-3: /var/log
raw-size: 232.88 GiB
size: 232.88 GiB (100.00%)
used: 90.71 GiB (39.0%)
fs: btrfs
dev: /dev/dm-0
maj-min: 254:0
mapped: luks-ef44cf91-5209-4ff4-95bf-ffb124a71d7e
Swap:
Kernel:
swappiness: 1 (default 60)
cache-pressure: 100 (default)
ID-1: swap-1
type: file
size: 512 MiB
used: 1.3 MiB (0.2%)
priority: -2
file: /swap/swapfile
Sensors:
System Temperatures:
cpu: 53.0 C
mobo: N/A
Fan Speeds (RPM): N/A
Info:
Processes: 219
Uptime: 14m
wakeups: 5
Memory: 3.8 GiB
used: 2.7 GiB (71.1%)
Init: systemd
v: 250
tool: systemctl
Compilers:
gcc: 11.1.0
clang: 13.0.0
Packages:
pacman: 1249
lib: 334
flatpak: 0
Shell: Zsh
v: 5.8
default: Bash
v: 5.1.16
running-in: konsole
inxi: 3.3.12
```
</details>
I got 2 system hangs while pamac doing that brand new `Refreshing AUR` stage: last one I have a web browser opened with about 5 youtube video tabs, at least three of them was paused, one was watching, also I have as opened a one more tab.
I decide to issue the pamac command above.
I got unresponsive PC at least for 1h 20 mins. After that I pulled out power source from the PC.
That was second time with that command run and on that mentioned stage.
After reboot I split terminal screen into two parts and executed the
```text
watch -n 1 free -m
```
and
```text
pamac update --force-refresh
```
I watched for the `available` field value of `free`.
For some AUR procedure I saw pamac was downloading a 9.3 MiB file and then was the `Refreshing AUR...` stage. I saw that while a file was downloading RAM `available` value was near 2400 MiB, during the stage I saw it was as low as 980 MiB.
I think that low end PCs (with low memory is incompatible with the current algorithm of `pamac`'s force refresh action).
The issue is absent while I update other repos, including `chaotic-aur` but present during that `Refreshing AUR` stage.
Please try to optimize the algorithm to lower RAM usage (1.5 GiB from 10 MiB file) or add `pamac` requirements of 1.5 of RAM to proceed the dbs force update request.
Thank you!https://gitlab.manjaro.org/applications/pamac/-/issues/1171Show note when AUR is not available2022-02-20T09:12:47ZTobias GeroldShow note when AUR is not available## Pamac Version
10.3.0
## Libpamac Version
11.2.0
## Desktop environment
KDE
## Description
I would like to be notified if AUR is not available.## Pamac Version
10.3.0
## Libpamac Version
11.2.0
## Desktop environment
KDE
## Description
I would like to be notified if AUR is not available.https://gitlab.manjaro.org/applications/pamac/-/issues/1169Feature Request: Change AUR URL to reverse proxy2022-02-20T09:13:01Z19000116041900011604@pku.edu.cnFeature Request: Change AUR URL to reverse proxy## Pamac Version
10.2.0
## Libpamac Version
11.1.0
## Desktop environment
KDE
In China, due to the Great Firewall, the link to AUR is bad and cause problems:
```
pamac build (package_name)
Failed to query https://aur.archlinux.org/rp...## Pamac Version
10.2.0
## Libpamac Version
11.1.0
## Desktop environment
KDE
In China, due to the Great Firewall, the link to AUR is bad and cause problems:
```
pamac build (package_name)
Failed to query https://aur.archlinux.org/rpc/?v=5&type=info&arg[]=(package_name) from AUR:
Socket I/O timed out
Error: target not found: (package_name)
```
But there is a reverse proxy https://aur.tuna.tsinghua.edu.cn which is fast in China, which can use by `yay` via `yay --aururl "https://aur.tuna.tsinghua.edu.cn" --save`, I think it is useful for pamac to add this feature.https://gitlab.manjaro.org/applications/pamac/-/issues/1161AUR Metadata Archive (live)2022-02-20T09:23:21ZKevin MorrisAUR Metadata Archive (live)Hello Pamac developers,
We (the AUR development team) are in the process of finalizing a new metadata archive. This kind of archive gives people the ability to search without actually using the RPC `type=search` API and reducing their r...Hello Pamac developers,
We (the AUR development team) are in the process of finalizing a new metadata archive. This kind of archive gives people the ability to search without actually using the RPC `type=search` API and reducing their requests toward the AUR overall. I have attempted to reach out to one of your developers, however, I wanted to create this issue here so that the project in general is aware and can give more feedback.
Currently, we are considering three slightly different features, a metadata archive containing:
1. (a) RPC `type=search` formatted JSON data
- Size: roughly 2.1MB
2. (a) Partial RPC `type=multiinfo` formatted JSON data
- Size: roughly 3-6MB (haven't verified this size yet, as we haven't done the modification of the multiinfo format)
- Contains extended fields: License and Keywords
3. (b) RPC `type=multiinfo` formatted JSON data
- Size: roughly 9.5MB
- Contains extended fields: License, Keywords, all dependencies and package relationships.
The different features are also annotated with an (a) and (b), signifying that we want to make a selection so that there are is only one annotation remaining. Apologies that this... is really organized in a bit of an ugly way here, but I believe this should convey what's going on.
Off the bat, I believe it would be pretty useful for all projects which search through the AUR to have access to 1. This removes the need for API users to utilize the `type=search` API on local machines when they know they will not be performing a simple request, completely removing the `type=search` utilization that Pamac may currently use from the AUR's resources.
However, we do believe there may be need for these other extended forms. That being said, a 9.5MB (3) archive might be a bit rough for a client to download; hence, a middle of choice both: 2. This would also include the `License` and `Keywords` fields that the `type=multiinfo` API provides, but not include related packages or dependencies.
In regards to 3, we may also provide this type of format for those who really desire it, but as an extra archive along with the base metadata archive we wish to host. At least, then, users of the archives would have a choice about how detailed they want the archive to be.
All archives on the AUR give out the `Last-Modified` header and support the `If-Modified-Since` header, which should be populated with `Last-Modified` on subsequent requests.
Sidenote: We really do not want to extend archives past what the RPC offers currently, as we believe the RPC offerings should be able to fulfill client needs as it is. However, if there is a good reason to do so, we may consider adding them to the RPC and these archives in general.
So, I have two questions here:
- Out of the current features, which ones do you believe would serve Pamac the best, and why?
- What kind of details would be useful that the current features listed do not have, and why?
Your feedback here is greatly appreciated; we are really trying to come up with a solution that will calm down Pamac's querying toward the AUR, but still give it the ability to perform as it wants.https://gitlab.manjaro.org/applications/pamac/-/issues/1057[Feature Request] Add option to automatically remove make dependencies2022-11-23T18:33:30ZOscar Watton[Feature Request] Add option to automatically remove make dependenciesIt would be very convenient to have an option to automatically remove make dependencies after building AUR packages, so users don't have to manually remove orphans every time (I believe I'm not the only one who does that).
Some other AU...It would be very convenient to have an option to automatically remove make dependencies after building AUR packages, so users don't have to manually remove orphans every time (I believe I'm not the only one who does that).
Some other AUR helpers (yay, paru) already have this option.https://gitlab.manjaro.org/applications/pamac/-/issues/831[Feature Request] Manage AUR votes2022-02-20T09:18:22Zsomepaulo[Feature Request] Manage AUR votesPlease introduce a way to manage AUR votes either through aurvote-utils or otherwise.
Thank you!Please introduce a way to manage AUR votes either through aurvote-utils or otherwise.
Thank you!