Problem:

When a kernel module is compiled as a separate module, some important
information about the kernel module is available via .modinfo section of
the module.  In contrast, when the kernel module is compiled into the
kernel, that information is not available.

Information about built-in modules is necessary in the following cases:

1. When it is necessary to find out what additional parameters can be
passed to the kernel at boot time.

2. When you need to know which module names and their aliases are in
the kernel. This is very useful for creating an initrd image.

Proposal:

The proposed patch does not remove .modinfo section with module
information from the vmlinux at the build time and saves it into a
separate file after kernel linking. So, the kernel does not increase in
size and no additional information remains in it. Information is stored
in the same format as in the separate modules (null-terminated string
array). Because the .modinfo section is already exported with a separate
modules, we are not creating a new API.

It can be easily read in the userspace:

$ tr '\0' '\n' < modules.builtin.modinfo
ext4.softdep=pre: crc32c
ext4.license=GPL
ext4.description=Fourth Extended Filesystem
ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, Theodore Ts'o and others
ext4.alias=fs-ext4
ext4.alias=ext3
ext4.alias=fs-ext3
ext4.alias=ext2
ext4.alias=fs-ext2
md_mod.alias=block-major-9-*
md_mod.alias=md
md_mod.description=MD RAID framework
md_mod.license=GPL
md_mod.parmtype=create_on_open:bool
md_mod.parmtype=start_dirty_degraded:int
...

Co-Developed-by: Gleb Fotengauer-Malinovskiy <glebfm@altlinux.org>
Signed-off-by: Gleb Fotengauer-Malinovskiy <glebfm@altlinux.org>
Signed-off-by: Alexey Gladkov <gladkov.alexey@gmail.com>
Acked-by: Jessica Yu <jeyu@kernel.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

A minor code cleanup.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Handle samples/ like the other top-level directories to simplify
the Makefile.

Include include/config/auto.conf earlier to evaluate
drivers-$(CONFIG_SAMPLES).

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

We already did this for clang, but now gcc has that warning too.  Yes,
yes, the address may be unaligned.  And that's kind of the point.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

CONFIG_INIT_STACK_ALL turns on stack initialization based on
-ftrivial-auto-var-init in Clang builds, which has greater coverage
than CONFIG_GCC_PLUGINS_STRUCTLEAK_BYREF_ALL.

-ftrivial-auto-var-init Clang option provides trivial initializers for
uninitialized local variables, variable fields and padding.

It has three possible values:
  pattern - uninitialized locals are filled with a fixed pattern
    (mostly 0xAA on 64-bit platforms, see https://reviews.llvm.org/D54604


    for more details, but 0x000000AA for 32-bit pointers) likely to cause
    crashes when uninitialized value is used;
  zero (it's still debated whether this flag makes it to the official
    Clang release) - uninitialized locals are filled with zeroes;
  uninitialized (default) - uninitialized locals are left intact.

This patch uses only the "pattern" mode when CONFIG_INIT_STACK_ALL is
enabled.

Developers have the possibility to opt-out of this feature on a
per-variable basis by using __attribute__((uninitialized)), but such
use should be well justified in comments.

Co-developed-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Kees Cook <keescook@chromium.org>
Tested-by: Alexander Potapenko <glider@google.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>

After cross-compiling the kernel, "make mrproper" should be executed
with the proper ARCH= option. Otherwise, stale objects will remain
under arch/$(SRCARCH)/.

One bad scenario is like this:

  $ make ARCH=arm defconfig all   # cross-compile the kernel for arm
  $ make mrproper                 # mrproper for host-arch (i.e. x86)
  $ make ARCH=arm O=build_dir defconfig all

If you miss ARCH= for mrproper and cross-compile the kernel with O=
and ARCH= options, Kbuild will happily start to build, but may fail
due to stale objects in the srctree.

If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop
the out-of-tree build. To detect this, mrproper should clean only
arch/$(SRCARCH)/include/generated/.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This is unneeded since commit 43fee2b2 ("kbuild: do not redirect
the first prerequisite for filechk").

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

GCC 9 introduces a new option, -flive-patching. It disables certain
optimizations which could make a compilation unsafe for later live
patching of the running kernel.

The option is used only if CONFIG_LIVEPATCH is enabled and $(CC)
supports it.

Performance impact of the option was measured on three different
Intel machines - two bigger NUMA boxes and one smaller UMA box. Kernel
intensive (IO, scheduling, networking) benchmarks were selected, plus a
set of HPC workloads from NAS Parallel Benchmark. The tests were done on
upstream kernel 5.0-rc8 with openSUSE Leap 15.0 userspace.

The majority of the tests is unaffected. The only significant exception
is the scheduler section which suffers 1-3% degradation.

Evaluated-by: Giovanni Gherdovich <ggherdovich@suse.cz>
Signed-off-by: Miroslav Benes <mbenes@suse.cz>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>

This patch adds new config option to trigger generation of BTF type
information from DWARF debuginfo for vmlinux and kernel modules through
pahole, which in turn relies on libbpf for btf_dedup() algorithm.

The intent is to record compact type information of all types used
inside kernel, including all the structs/unions/typedefs/etc. This
enables BPF's compile-once-run-everywhere ([0]) approach, in which
tracing programs that are inspecting kernel's internal data (e.g.,
struct task_struct) can be compiled on a system running some kernel
version, but would be possible to run on other kernel versions (and
configurations) without recompilation, even if the layout of structs
changed and/or some of the fields were added, removed, or renamed.

This is only possible if BPF loader can get kernel type info to adjust
all the offsets correctly. This patch is a first time in this direction,
making sure that BTF type info is part of Linux kernel image in
non-loadable ELF section.

BTF deduplication ([1]) algorithm typically provides 100x savings
compared to DWARF data, so resulting .BTF section is not big as is
typically about 2MB in size.

[0] http://vger.kernel.org/lpc-bpf2018.html#session-2
[1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html



Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

KBUILD_SRC was conventionally used for some different purposes:
 [1] To remember the source tree path
 [2] As a flag to check if sub-make is already done
 [3] As a flag to check if Kbuild runs out of tree

For [1], we do not need to remember it because the top Makefile
can compute it by $(realpath $(dir $(lastword $(MAKEFILE_LIST))))

[2] has been replaced with self-commenting 'sub_make_done'.

For [3], we can distinguish in-tree/out-of-tree by comparing
$(srctree) and '.'

This commit converts [3] to prepare for the KBUILD_SRC removal.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Kbuild always runs in the top of the output directory.

If Make starts in the source directory with O=, it relocates the
working directory to the location specified by O=.

Also, users can start build from the output directory by using the
Makefile generated by scripts/mkmakefile.

With a little more effort, Kbuild will be able to start from any
directory path.

This commit allows to specify the source directory by using
the -f option.

For example, you can do:

  $ cd path/to/output/dir
  $ make -f path/to/source/dir/Makefile

Or, for the equivalent behavior, you can do:

  $ make O=path/to/output/dir -f path/to/source/dir/Makefile

KBUILD_SRC is now deprecated.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kieran Bingham <kbingham@kernel.org>

Manipulating $(MAKECMDGOALS) for sub-make seems odd to me.

[1] 'make O=foo sub-make' is turned into 'make O=foo', which builds
the default targets. It would make sense to terminate the build with:

  *** No rule to make target 'sub-make'.  Stop.

[2] 'make O=foo defconfig _all' is turned into 'make O=foo defconfig',
which changes the behavior.

Let's pass $(MAKECMDGOALS) as is.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

If you do "make Makefile" with GNU Make 3.x, the following warning
is displayed:

  $ make Makefile
  Makefile:165: warning: overriding recipe for target 'Makefile'
  Makefile:51: warning: ignoring old recipe for target 'Makefile'
  make[1]: Nothing to be done for 'Makefile'.
  make: Nothing to be done for 'Makefile'.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

When you run a "make *config" target, the retpoline compiler flags
are evaluated for nothing because the code is located above the
'ifeq ($(config-targets),1)'.

Move it a bit below to avoid unneeded computation in the Kconfig stage.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Commit 3a51ff34 ("kbuild: gitignore output directory") seemed to
bother people who version-control output directories.

Andre Przywara says:
"Unfortunately this breaks my setup, because I keep a totally separate
git repository in my build directories to track (various versions of)
.config. So .gitignore there is carefully crafted to ignore most build
artefacts, but not .config, for instance."

Link: https://lkml.org/lkml/2019/3/22/1819


Reported-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>

When Make recurses to the top Makefile with sub-make-done unset,
the code block surrounded by 'ifneq ($(sub-make-done),1) ... endif'
is parsed multiple times. This happens for in-tree building of
include/config/auto.conf, *-pkg, etc. with GNU Make 4.x.

This is a slight regression by commit 688931a5 ("kbuild: skip
sub-make for in-tree build with GNU Make 4.x") in terms of performance
since that code block contains one $(shell ...) invocation.

Fix it by exporting the variable irrespective of sub-make being run.
I renamed it because GNU Make cannot properly export variables
containing hyphens. This is probably a bug of GNU Make, and the issue
in Kbuild had already been reported by commit 2bfbe788 ("kbuild:
Do not use hyphen in exported variable name").

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

If it is not in the default location, compilation fails at several points.

Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/91a25e992566a7968fedc89ec80e7f4c83ad0548.1553622500.git.jpoimboe@redhat.com

Commit 2b50f7ab ("kbuild: add workaround for Debian make-kpkg")
annoyed people who want to wrap the top Makefile with GNUmakefile
to customize it for their use.

On second thought, we do not need to run the sub-make for in-tree
build with Make 4.x because the 'MAKEFLAGS += -rR' issue only happens
on GNU Make 3.x.

With this commit, people will get back their workflow, and the Debian
make-kpkg will still work.

Fixes: 2b50f7ab ("kbuild: add workaround for Debian make-kpkg")
Reported-by: Andreas Schwab <schwab@suse.de>
Reported-by: David Howells <dhowells@redhat.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Andreas Schwab <schwab@suse.de>
Tested-by: David Howells <dhowells@redhat.com>

The clang option -Oz enables *aggressive* optimization for size,
which doesn't necessarily result in smaller images, but can have
negative impact on performance. Switch back to the less aggressive
-Os.

This reverts commit 6748cb3c.

Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Matthias Kaehlcke <mka@chromium.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, every arch/*/include/uapi/asm/Kbuild explicitly includes
the common Kbuild.asm file. Factor out the duplicated include directives
to scripts/Makefile.asm-generic so that no architecture would opt out
of the mandatory-y mechanism.

um is not forced to include mandatory-y since it is a very exceptional
case which does not support UAPI.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

During a simple no-op (nothing changed) build I saw 39 invocations of
the C compiler with the argument "-print-file-name=include".  We don't
need to call the C compiler 39 times for this--one time will suffice.

Let's change NOSTDINC_FLAGS to a simply expanded variable to avoid
this since there doesn't appear to be any reason it should be
recursively expanded.

On my build this shaved ~400 ms off my "no-op" build.

Note that the recursive expansion seems to date back to the (really
old) commit e8f5bdb0 ("[PATCH] Makefile include path ordering").
It's a little unclear to me if the point of that patch was to switch
the variable to be recursively expanded (which it did) or to avoid
directly assigning to NOSTDINC_FLAGS (AKA to switch to +=) because
someone else (out of tree?) was setting it.  I presume later since if
the only goal was to switch to recursive expansion the patch would
have just removed the ":".

Signed-off-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Since commit 3812b8c5 ("kbuild: make -r/-R effective in top
Makefile for old Make versions"), make-kpkg is not working.

make-kpkg directly includes the top Makefile of Linux kernel, and
appends some debian_* targets.

  /usr/share/kernel-package/ruleset/kernel_version.mk:

    # Include the kernel makefile
    override dot-config := 1
    include Makefile
    dot-config := 1

I did not know the kernel Makefile was used in that way, and it is
hard to guarantee the behavior when the kernel Makefile is included
by another Makefile from a different project.

It looks like Debian Stretch stopped providing make-kpkg. Maybe it is
obsolete and being replaced with 'make deb-pkg' etc. but still widely
used.

This commit adds a workaround; if the top Makefile is included by
another Makefile, skip sub-make in order to make the main part visible.
'MAKEFLAGS += -rR' does not become effective for GNU Make < 4.0, but
Debian/Ubuntu is already using newer versions.

The effect of this commit:

  Debian 8 (Jessie)  : Fixed
  Debian 9 (Stretch) : make-kpkg (kernel-package) is not provided
  Ubuntu 14.04 LTS   : NOT Fixed
  Ubuntu 16.04 LTS   : Fixed
  Ubuntu 18.04 LTS   : Fixed

This commit cannot fix Ubuntu 14.04 because it installs GNU Make 3.81,
but its support will end in Apr 2019, which is before the Linux v5.1
release.

I added warning so that nobody would try to include the top Makefile.

Fixes: 3812b8c5 ("kbuild: make -r/-R effective in top Makefile for old Make versions")
Reported-by: Liz Zhang <lizzha@microsoft.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Lili Deng <v-lide@microsoft.com>
Cc: Manoj Srivastava <srivasta@debian.org>

This build rule was introduced by commit cd05e6bd ("[PATCH]
kbuild: fix split-include dependency") to handle the dependency of
scripts/basic/split-include.

Now, fixdep is the only tool in scripts/basic/, and this rule is
no longer used.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The flag '-Werror-implicit-function-declaration', present in GCC 2.95,
stopped to be documented in GCC 4.3, replaced by the more generic
'-Werror=...' form.

So, use the equivalent '-Werror=implicit-function-declaration' instead.

Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This code has been commented out since commit b7000ade
("Don't set the INITRD_COMPRESS environment variable automatically").

Clean it up now.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

CONFIG_DEBUG_INFO_SPLIT and CONFIG_DEBUG_INFO_DWARF4 enable extra
dwarf options if supported. You never know if they are really enabled
since Makefile may silently turn them off.

The actual behavior will match to the kernel configuration by
testing those compiler flags in the Kconfig stage.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

If you run "make" in a pristine source tree, currently Kbuild will
start to build Kconfig to let it show the error message.

It would be more straightforward to check it in Makefile and let
it fail immediately.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

If include/config/auto.conf.cmd is lost for some reasons, it is not
self-healing, so the top Makefile misses to run syncconfig.
Move include/config/auto.conf.cmd to the target side.

I used a pattern rule instead of a normal rule here although it is
a bit gross.

If the rule were written with a normal rule like this,

  include/config/auto.conf \
  include/config/auto.conf.cmd \
  include/config/tristate.conf: $(KCONFIG_CONFIG)
          $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig

... syncconfig would be executed per target.

Using a pattern rule makes sure that syncconfig is executed just once
because Make assumes the recipe will create all of the targets.

Here is a quote from the GNU Make manual [1]:

"Pattern rules may have more than one target. Unlike normal rules,
this does not act as many different rules with the same prerequisites
and recipe. If a pattern rule has multiple targets, make knows that
the rule's recipe is responsible for making all of the targets. The
recipe is executed only once to make all the targets. When searching
for a pattern rule to match a target, the target patterns of a rule
other than the one that matches the target in need of a rule are
incidental: make worries only about giving a recipe and prerequisites
to the file presently in question. However, when this file's recipe is
run, the other targets are marked as having been updated themselves."

[1]: https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html



Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The dependency will be checked anyway after Kbuild descends into a
sub-directory. Skip object/source dependency checks in top Makefile.

VPATH can be simpler since the top Makefile no longer checks the
presence of the source file, which is located in in the external
module directory.

One good thing is, it can compile an object from a generated source
file.

  $ make crypto/rsapubkey.asn1.o
    ...
  ASN.1   crypto/rsapubkey.asn1.c
  CC      crypto/rsapubkey.asn1.o

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The previous commit made 'MAKEFLAGS += -rR' effective in the top
Makefile regardless of O= option, GNU Make versions.

The top Makefile does not need to cancel implicit rules for makefiles.

There is still one place where an empty rule is useful. Since -rR is
effective only after sub-make, GNU Make would try implicit rules to
update the top Makefile. Although it is not a big overhead, cancel it
just in case.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Adding -rR to MAKEFLAGS is important because we do not want to
be bothered by built-in implicit rules or variables.

One problem that used to exist in older GNU Make versions is

  MAKEFLAGS += -rR

... does not become effective in the current Makefile. When you are
building with O= option, it becomes effective in the top Makefile
since it recurses via 'sub-make' target. Otherwise, the top Makefile
tries implicit rules. That is why we explicitly add empty rules for
Makefiles, but we often miss to do that.

In fact, adding -d option to older GNU Make versions shows it is
trying a bunch of implicit pattern rules.

 Considering target file `scripts/Makefile.kcov'.
  Looking for an implicit rule for `scripts/Makefile.kcov'.
  Trying pattern rule with stem `Makefile.kcov'.
  Trying implicit prerequisite `scripts/Makefile.kcov.o'.
  Trying pattern rule with stem `Makefile.kcov'.
  Trying implicit prerequisite `scripts/Makefile.kcov.c'.
  Trying pattern rule with stem `Makefile.kcov'.
  Trying implicit prerequisite `scripts/Makefile.kcov.cc'.
  Trying pattern rule with stem `Makefile.kcov'.
  Trying implicit prerequisite `scripts/Makefile.kcov.C'.
  ...

This issue was fixed by GNU Make commit 58dae243526b ("[Savannah #20501]
Handle adding -r/-R to MAKEFLAGS in the makefile"). So, it is no longer
a problem if you use GNU Make 4.0 or later. However, older versions are
still widely used.

So, I decided to patch the kernel Makefile to invoke sub-make regardless
of O= option. This will allow further cleanups.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>