From 7662b6c27ed76533bce8139edbe983f9bfb020c6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 10 Aug 2017 08:47:44 +0200 Subject: [PATCH 001/310] usb: Add UQ_MSC_NO_INQUIRY quirk for external "Intenso Memory" disk Without the quirk the disk disappears shortly after beeing detected: 08:36:35 t520 kernel: ugen1.4: at usbus1 08:36:36 t520 kernel: umass0 on uhub3 08:36:36 t520 kernel: umass0: on usbus1 08:36:36 t520 kernel: umass0: SCSI over Bulk-Only; quirks = 0x0100 08:36:36 t520 kernel: umass0:5:0: Attached to scbus5 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): REPORT LUNS. CDB: a0 00 00 00 00 00 00 00 00 10 00 00 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: SCSI Status Error 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): SCSI status: Check Condition 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code) 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): Error 22, Unretryable error 08:37:49 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:37:49 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:37:49 t520 kernel: (probe0:umass-sim0:0:0:0): Retrying command 08:37:54 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:37:54 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:37:54 t520 kernel: (probe0:umass-sim0:0:0:0): Retrying command 08:38:00 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:38:00 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:38:00 t520 kernel: (probe0:umass-sim0:0:0:0): Retrying command 08:38:05 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:38:05 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:38:05 t520 kernel: (probe0:umass-sim0:0:0:0): Retrying command 08:38:11 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:38:11 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:38:11 t520 kernel: (probe0:umass-sim0:0:0:0): Error 5, Retries exhausted 08:38:44 t520 kernel: (da0:umass-sim0:0:0:0): got CAM status 0x44 08:38:44 t520 kernel: (da0:umass-sim0:0:0:0): fatal error, failed to attach to device 08:38:44 t520 kernel: g_access(918): provider da0 has error 6 set 08:38:44 t520 last message repeated 2 times fk@t520 ~ $sudo usbconfig -u 1 -a 4 dump_device_desc ugen1.4: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (2mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0210 bDeviceClass = 0x0000 bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0040 idVendor = 0x2109 idProduct = 0x0701 bcdDevice = 0x0307 iManufacturer = 0x0001 iProduct = 0x0002 iSerialNumber = 0x0003 <20130519000569> bNumConfigurations = 0x0001 PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221852 PR submission date: 2017-08-27 Obtained from: ElectroBSD --- sys/dev/usb/quirk/usb_quirk.c | 1 + sys/dev/usb/usbdevs | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/sys/dev/usb/quirk/usb_quirk.c b/sys/dev/usb/quirk/usb_quirk.c index d86ca5643515..6517437a8ae4 100644 --- a/sys/dev/usb/quirk/usb_quirk.c +++ b/sys/dev/usb/quirk/usb_quirk.c @@ -275,6 +275,7 @@ static struct usb_quirk_entry usb_quirks[USB_DEV_QUIRKS_MAX] = { USB_QUIRK(INSYSTEM, USBCABLE, 0x0000, 0xffff, UQ_MSC_FORCE_WIRE_CBI, UQ_MSC_FORCE_PROTO_ATAPI, UQ_MSC_NO_TEST_UNIT_READY, UQ_MSC_NO_START_STOP, UQ_MSC_ALT_IFACE_1), + USB_QUIRK(INTENSO, MEMORY_BOX, 0x0000, 0xffff, UQ_MSC_NO_INQUIRY), USB_QUIRK(INSYSTEM, ATAPI, 0x0000, 0xffff, UQ_MSC_FORCE_WIRE_CBI, UQ_MSC_FORCE_PROTO_RBC), USB_QUIRK(INSYSTEM, STORAGE_V2, 0x0000, 0xffff, UQ_MSC_FORCE_WIRE_CBI, diff --git a/sys/dev/usb/usbdevs b/sys/dev/usb/usbdevs index b07be3edd241..032c6be1fbfe 100644 --- a/sys/dev/usb/usbdevs +++ b/sys/dev/usb/usbdevs @@ -768,7 +768,8 @@ vendor SIMTEC 0x20df Simtec Electronics vendor TRENDNET 0x20f4 TRENDnet vendor RTSYSTEMS 0x2100 RT Systems vendor DLINK4 0x2101 D-Link -vendor VIALABS 0x2109 VIA Labs +vendor VIALABS 0x2109 VIA Lab +vendor INTENSO 0x2109 INTENSO vendor ERICSSON 0x2282 Ericsson vendor MOTOROLA2 0x22b8 Motorola vendor WETELECOM 0x22de WeTelecom @@ -2598,6 +2599,9 @@ product INSYSTEM ISD105 0x0202 IDE Adapter ISD105 product INSYSTEM USBCABLE 0x081a USB cable product INSYSTEM STORAGE_V2 0x5701 USB Storage Adapter V2 +/* Intenso products */ +product INTENSO MEMORY_BOX 0x0701 External disk + /* Intel products */ product INTEL EASYPC_CAMERA 0x0110 Easy PC Camera product INTEL TESTBOARD 0x9890 82930 test board -- 2.37.1 From 3e2b7efea703c83aa5ae16f6bc7b7b8cfb371d4b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 21 May 2017 14:09:13 +0200 Subject: [PATCH 002/310] libdtrace: Prevent an assertion from triggering with certain llquantize() parameters The dtrace command that reached the assertion: fk@t520 ~ $sudo dtrace -n 'syscall::read:return /execname == "privoxy"/ { @[execname] = llquantize(arg0, 100, 0, 10, 100); @m = max(arg0)}' [...] 9800 | 0 9900 | 0 10000 |@@@@@@@@@@@@@@@@@@@@ 37 20000 | 0 Assertion failed: (value < next), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c, line 1083. Abort trap (gdb) where #0 0x00000008011effda in thr_kill () from /lib/libc.so.7 #1 0x00000008011effa4 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52 #2 0x00000008011eff19 in abort () at /usr/src/lib/libc/stdlib/abort.c:65 #3 0x000000080088c3b2 in __assert (expr=0x8008d3172 "value < next", file=0x8008d3078 "/usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c", line=1083) at /usr/src/cddl/lib/libdtrace/../../../cddl/compat/opensolaris/include/assert.h:56 #4 0x000000080088c190 in dt_print_llquantize (dtp=0x802633000, fp=0x8014c37e8, addr=0x80269a110, size=7840, normal=1) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:1083 #5 0x000000080088e37d in dt_print_datum (dtp=0x802633000, fp=0x8014c37e8, rec=0x8026900e8, addr=0x80269a110 "d", size=7848, aggdata=0x802690150, normal=1, pd=0x7fffffffe750) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:2211 #6 0x000000080088dc12 in dt_print_aggs (aggsdata=0x7fffffffe630, naggvars=1, arg=0x7fffffffe750) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:2313 #7 0x000000080088e6cf in dt_print_agg (aggdata=0x802690150, arg=0x7fffffffe750) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:2361 #8 0x0000000800895f8b in dt_aggregate_walk_sorted (dtp=0x802633000, func=0x80088e610 , arg=0x7fffffffe750, sfunc=0x0) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_aggregate.c:1585 #9 0x0000000800895d39 in dtrace_aggregate_walk_sorted (dtp=0x802633000, func=0x80088e610 , arg=0x7fffffffe750) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_aggregate.c:1605 #10 0x0000000800897f12 in dtrace_aggregate_print (dtp=0x802633000, fp=0x8014c37e8, func=0x800895d10 ) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_aggregate.c:2130 #11 0x0000000000403a5e in main (argc=, argv=) at /usr/src/cddl/usr.sbin/dtrace/../../../cddl/contrib/opensolaris/cmd/dtrace/dtrace.c:2005 (gdb) f 4 #4 0x000000080088c190 in dt_print_llquantize (dtp=0x802633000, fp=0x8014c37e8, addr=0x80269a110, size=7840, normal=1) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:1083 1083 assert(value < next); (gdb) p step $2915 = 77662796314522419 (gdb) p value $2916 = 7834326075677972872 (gdb) p next $2917 = 7766279631452241920 It works as expected when replacing the 10 with a 5. Various other parameter combinations work as expected as well and I've used similar commands for weeks without issues. The problem is reproducible with other execnames as long as the probe fires. The "@m = max(arg0)" part isn't required to trigger the assertion but I only noticed it after already patching the system where libdtrace is build with reduced optimizations. This commit may not be the best solution but it seems to reliably prevent the core dumps and the output continues to look reasonable. The code flow in dt_print_llquantize() seems strange to me and maybe the loop should break if "bin" reaches "last_bin" instead. My impression is that it does a bunch of cycles at the end without doing meaningful work. PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219451 PR submission date: 2017-05-22 Superior upstream commit in HEAD: r322773/95f65483b371f Obtained from: ElectroBSD --- cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c index 862d4b9222d7..7311f5340a46 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c @@ -1084,7 +1084,7 @@ dt_print_llquantize(dtrace_hdl_t *dtp, FILE *fp, const void *addr, assert(value < next); bin++; - if ((value += step) != next) + if ((value += step) < next) continue; next = value * factor; -- 2.37.1 From aa27b1ad0198bbc4b33091a66e2c8ed80dfe5e11 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 23 Aug 2017 08:52:11 +0200 Subject: [PATCH 003/310] share/dtrace: Stop pretending that the DTraceToolkit scripts can be installed ... by setting MK_CDDL. The DTraceToolkit scripts were removed in r300226 and now have to be installed through other means. Obtained from: ElectroBSD --- share/dtrace/Makefile | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 45b173565796..8ae7aad56fa5 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -1,7 +1,6 @@ # $FreeBSD$ # -# Hand installing our scripts and optionally (based on MK_CDDL) installing -# the DTraceToolkit. +# Hand installing our scripts. # .include -- 2.37.1 From 881e1c6f88c9a914b51f35f772aba7187eac6a70 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 19 Mar 2017 17:39:06 +0100 Subject: [PATCH 004/310] contrib/mtree: Fix small memory leak in spec() ... that was found when using makefs with an mtree spec: [fk@electrobsd /tmp/image-comparison] $valgrind --leak-check=full --tool=memcheck makefs -T 0 -B little -o label=blafasel -F mtree.spec image.part . ==35506== Memcheck, a memory error detector ==35506== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==35506== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info ==35506== Command: makefs -T 0 -B little -o label=blafasel -F mtree.spec image.part . ==35506== Calculated size of `image.part': 1917493248 bytes, 15 inodes Extent size set to 8192 density reduced from 127832884 to 907060 image.part: 1828.7MB (3745104 sectors) block size 8192, fragment size 1024 using 34 cylinder groups of 55.31MB, 7080 blks, 64 inodes. super-block backups (for fsck -b #) at: 32, 113312, 226592, 339872, 453152, 566432, 679712, 792992, 906272, 1019552, 1132832, 1246112, 1359392, 1472672, 1585952, 1699232, 1812512, 1925792, 2039072, 2152352, 2265632, 2378912, 2492192, 2605472, 2718752, 2832032, 2945312, 3058592, 3171872, 3285152, 3398432, 3511712, 3624992, 3738272, Populating `image.part' Image `image.part' complete ==35506== ==35506== HEAP SUMMARY: ==35506== in use at exit: 1,137,370 bytes in 271 blocks ==35506== total heap usage: 433,879 allocs, 433,608 frees, 1,792,473,254 bytes allocated ==35506== ==35506== 62 bytes in 1 blocks are definitely lost in loss record 1 of 15 ==35506== at 0x4C25DBE: realloc (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so) ==35506== by 0x41760A: spec (in /usr/sbin/makefs) ==35506== by 0x40BD90: apply_specfile (in /usr/sbin/makefs) ==35506== by 0x4090A7: main (in /usr/sbin/makefs) ==35506== ==35506== LEAK SUMMARY: ==35506== definitely lost: 62 bytes in 1 blocks ==35506== indirectly lost: 0 bytes in 0 blocks ==35506== possibly lost: 0 bytes in 0 blocks ==35506== still reachable: 1,137,308 bytes in 270 blocks ==35506== suppressed: 0 bytes in 0 blocks ==35506== Reachable blocks (those to which a pointer was found) are not shown. ==35506== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==35506== ==35506== For counts of detected and suppressed errors, rerun with: -v ==35506== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) Obtained from: ElectroBSD --- contrib/mtree/spec.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/mtree/spec.c b/contrib/mtree/spec.c index 6fbe6f534560..25ad285aec00 100644 --- a/contrib/mtree/spec.c +++ b/contrib/mtree/spec.c @@ -117,7 +117,7 @@ spec(FILE *fp) root = NULL; centry = last = NULL; - tname = NULL; + ntname = tname = NULL; tnamelen = 0; memset(&ginfo, 0, sizeof(ginfo)); for (mtree_lineno = 0; @@ -261,6 +261,7 @@ noparent: mtree_err("no parent node"); last = centry; } } + free(ntname); return (root); } -- 2.37.1 From 9e005277fb3e3660ae2fb42bad9e3a0aaffcbd9b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 23 Feb 2015 11:38:09 +0100 Subject: [PATCH 005/310] sys/conf/newvers.sh: Set TYPE to ElectroBSD Obtained from: ElectroBSD --- sys/conf/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) mode change 100644 => 100755 sys/conf/newvers.sh diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh old mode 100644 new mode 100755 index 48320ce1b7c3..8abbea51a0d5 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -52,7 +52,7 @@ # like the -V command # -TYPE="FreeBSD" +TYPE="ElectroBSD" REVISION="13.1" BRANCH="STABLE" if [ -n "${BRANCH_OVERRIDE}" ]; then -- 2.37.1 From 1f041685338a63d7be1179e59b6807bf4e05e66c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Sep 2015 14:34:45 +0200 Subject: [PATCH 006/310] sys/conf/newvers.sh: Do not add git hash derived from .git if KERNEL_VERSION_NUMBER is defined ... as this indicates that we are (trying to) reproduce a build. Do not use the REPRO_SEED for this as it's always set nowadays. XXX: There probably is prettier solution to do this. Obtained from: ElectroBSD --- sys/conf/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index 8abbea51a0d5..6ffb59fbe23a 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -216,7 +216,7 @@ if [ -z "${svnversion}" ] && [ -x /usr/bin/svnliteversion ] ; then fi fi -if findvcs .git; then +if [ -z "${KERNEL_VERSION_NUMBER}" ] && findvcs .git; then for dir in /usr/bin /usr/local/bin; do if [ -x "${dir}/git" ] ; then git_cmd="${dir}/git -c help.autocorrect=0 --git-dir=${VCSDIR}" -- 2.37.1 From 0e5618f5deb133c9508c226fde35e4edea5379cd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 28 May 2015 15:04:48 +0200 Subject: [PATCH 007/310] sys/conf/newvers.sh: Allow to overwrite the kernel version ... as the object directory may be dirty. Obtained from: ElectroBSD --- sys/conf/newvers.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index 6ffb59fbe23a..d9150ca83376 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -176,7 +176,7 @@ then fi touch version -v=$(cat version) +v=${KERNEL_VERSION_NUMBER:-`cat version`} u=${USER:-root} d=$(pwd) h=${HOSTNAME:-$(hostname)} @@ -318,4 +318,6 @@ if [ "$vers_content_new" != "$vers_content_old" ]; then printf "%s\n" "$vers_content_new" > vers.c fi -echo $((v + 1)) > version +if [ -z "${KERNEL_VERSION_NUMBER}" ]; then + echo $((v + 1)) > version +fi -- 2.37.1 From 1b76fdfaa1514d703e8e405da742bf032c2d26d5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2015 17:24:47 +0200 Subject: [PATCH 008/310] amd64: Save a copy of GENERIC as ELECTRO_BLOAT Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 365 +++++++++++++++++++++++++++++++++++ 1 file changed, 365 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BLOAT diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT new file mode 100644 index 000000000000..3d2cc39c6353 --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -0,0 +1,365 @@ +# +# GENERIC -- Generic kernel configuration file for FreeBSD/amd64 +# +# For more information on this file, please read the config(5) manual page, +# and/or the handbook section on Kernel Configuration Files: +# +# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html +# +# The handbook is also available locally in /usr/share/doc/handbook +# if you've installed the doc distribution, otherwise always see the +# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the +# latest information. +# +# An exhaustive list of options and more detailed explanations of the +# device lines is also present in the ../../conf/NOTES and NOTES files. +# If you are in doubt as to the purpose or necessity of a line, check first +# in NOTES. +# +# $FreeBSD$ + +cpu HAMMER +ident GENERIC + +makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols +makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support + +options SCHED_ULE # ULE scheduler +options PREEMPTION # Enable kernel thread preemption +options INET # InterNETworking +options INET6 # IPv6 communications protocols +options TCP_OFFLOAD # TCP offload +options SCTP # Stream Control Transmission Protocol +options FFS # Berkeley Fast Filesystem +options SOFTUPDATES # Enable FFS soft updates support +options UFS_ACL # Support for access control lists +options UFS_DIRHASH # Improve performance on big directories +options UFS_GJOURNAL # Enable gjournal-based UFS journaling +options QUOTA # Enable disk quotas for UFS +options MD_ROOT # MD is a potential root device +options NFSCL # Network Filesystem Client +options NFSD # Network Filesystem Server +options NFSLOCKD # Network Lock Manager +options NFS_ROOT # NFS usable as /, requires NFSCL +options MSDOSFS # MSDOS Filesystem +options CD9660 # ISO 9660 Filesystem +options PROCFS # Process filesystem (requires PSEUDOFS) +options PSEUDOFS # Pseudo-filesystem framework +options GEOM_PART_GPT # GUID Partition Tables. +options GEOM_RAID # Soft RAID functionality. +options GEOM_LABEL # Provides labelization +options COMPAT_FREEBSD32 # Compatible with i386 binaries +options COMPAT_FREEBSD4 # Compatible with FreeBSD4 +options COMPAT_FREEBSD5 # Compatible with FreeBSD5 +options COMPAT_FREEBSD6 # Compatible with FreeBSD6 +options COMPAT_FREEBSD7 # Compatible with FreeBSD7 +options COMPAT_FREEBSD9 # Compatible with FreeBSD9 +options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI +options KTRACE # ktrace(1) support +options STACK # stack(9) support +options SYSVSHM # SYSV-style shared memory +options SYSVMSG # SYSV-style message queues +options SYSVSEM # SYSV-style semaphores +options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions +options PRINTF_BUFR_SIZE=128 # Prevent printf output being interspersed. +options KBD_INSTALL_CDEV # install a CDEV entry in /dev +options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(4) +options AUDIT # Security event auditing +options CAPABILITY_MODE # Capsicum capability mode +options CAPABILITIES # Capsicum capabilities +options MAC # TrustedBSD MAC Framework +options KDTRACE_FRAME # Ensure frames are compiled in +options KDTRACE_HOOKS # Kernel DTrace hooks +options DDB_CTF # Kernel ELF linker loads CTF data +options INCLUDE_CONFIG_FILE # Include this file in kernel +options RACCT # Resource accounting framework +options RACCT_DEFAULT_TO_DISABLED # Set kern.racct.enable=0 by default +options RCTL # Resource limits + +# Debugging support. Always need this: +options KDB # Enable kernel debugger support. +options KDB_TRACE # Print a stack trace for a panic. +# For full debugger support use (turn off in stable branch): +options DDB # Support DDB. +options GDB # Support remote GDB. +options DEADLKRES # Enable the deadlock resolver +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones + +# Make an SMP-capable kernel by default +options SMP # Symmetric MultiProcessor Kernel + +# CPU frequency control +device cpufreq + +# Bus support. +device acpi +options ACPI_DMAR +device pci +options PCI_IOV # PCI SR-IOV support + +# Floppy drives +device fdc + +# ATA controllers +device ahci # AHCI-compatible SATA controllers +device ata # Legacy ATA/SATA controllers +options ATA_STATIC_ID # Static device numbering +device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA +device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA + +# SCSI Controllers +device ahc # AHA2940 and onboard AIC7xxx devices +options AHC_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~128k to driver. +device ahd # AHA39320/29320 and onboard AIC79xx devices +options AHD_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~215k to driver. +device esp # AMD Am53C974 (Tekram DC-390(T)) +device hptiop # Highpoint RocketRaid 3xxx series +device isp # Qlogic family +#device ispfw # Firmware for QLogic HBAs- normally a module +device mpt # LSI-Logic MPT-Fusion +device mps # LSI-Logic MPT-Fusion 2 +device mpr # LSI-Logic MPT-Fusion 3 +#device ncr # NCR/Symbios Logic +device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') +device trm # Tekram DC395U/UW/F DC315U adapters + +device adv # Advansys SCSI adapters +device adw # Advansys wide SCSI adapters +device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. +device bt # Buslogic/Mylex MultiMaster SCSI adapters +device isci # Intel C600 SAS controller + +# ATA/SCSI peripherals +device scbus # SCSI bus (required for ATA/SCSI) +device ch # SCSI media changers +device da # Direct Access (disks) +device sa # Sequential Access (tape etc) +device cd # CD +device pass # Passthrough device (direct ATA/SCSI access) +device ses # Enclosure Services (SES and SAF-TE) +#device ctl # CAM Target Layer + +# RAID controllers interfaced to the SCSI subsystem +device amr # AMI MegaRAID +device arcmsr # Areca SATA II RAID +device ciss # Compaq Smart RAID 5* +device dpt # DPT Smartcache III, IV - See NOTES for options +device hptmv # Highpoint RocketRAID 182x +device hptnr # Highpoint DC7280, R750 +device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx +device hpt27xx # Highpoint RocketRAID 27xx +device iir # Intel Integrated RAID +device ips # IBM (Adaptec) ServeRAID +device mly # Mylex AcceleRAID/eXtremeRAID +device twa # 3ware 9000 series PATA/SATA RAID +device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller + +# RAID controllers +device aac # Adaptec FSA RAID +device aacp # SCSI passthrough for aac (requires CAM) +device aacraid # Adaptec by PMC RAID +device ida # Compaq Smart RAID +device mfi # LSI MegaRAID SAS +device mlx # Mylex DAC960 family +device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s +#XXX pointer/int warnings +#device pst # Promise Supertrak SX6000 +device twe # 3ware ATA RAID + +# NVM Express (NVMe) support +device nvme # base NVMe driver +device nvd # expose NVMe namespaces as disks, depends on nvme + +# atkbdc0 controls both the keyboard and the PS/2 mouse +device atkbdc # AT keyboard controller +device atkbd # AT keyboard +device psm # PS/2 mouse + +device kbdmux # keyboard multiplexer + +device vga # VGA video card driver +options VESA # Add support for VESA BIOS Extensions (VBE) + +device splash # Splash screen and screen saver support + +# syscons is the default console driver, resembling an SCO console +device sc +options SC_PIXEL_MODE # add support for the raster text mode + +# vt is the new video console driver +device vt +device vt_vga +device vt_efifb + +device agp # support several AGP chipsets + +# PCCARD (PCMCIA) support +# PCMCIA and cardbus bridge support +device cbb # cardbus (yenta) bridge +device pccard # PC Card (16-bit) bus +device cardbus # CardBus (32-bit) bus + +# Serial (COM) ports +device uart # Generic UART driver + +# Parallel port +device ppc +device ppbus # Parallel port bus (required) +device lpt # Printer +device ppi # Parallel port interface device +#device vpo # Requires scbus and da + +device puc # Multi I/O cards and multi-channel UARTs + +# PCI Ethernet NICs. +device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE +device de # DEC/Intel DC21x4x (``Tulip'') +device em # Intel PRO/1000 Gigabit Ethernet Family +device igb # Intel PRO/1000 PCIE Server Gigabit Family +device ix # Intel PRO/10GbE PCIE PF Ethernet +device ixv # Intel PRO/10GbE PCIE VF Ethernet +device ixl # Intel XL710 40Gbe PCIE Ethernet +device ixlv # Intel XL710 40Gbe VF PCIE Ethernet +device le # AMD Am7900 LANCE and Am79C9xx PCnet +device ti # Alteon Networks Tigon I/II gigabit Ethernet +device txp # 3Com 3cR990 (``Typhoon'') +device vx # 3Com 3c590, 3c595 (``Vortex'') + +# PCI Ethernet NICs that use the common MII bus controller code. +# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! +device miibus # MII bus support +device ae # Attansic/Atheros L2 FastEthernet +device age # Attansic/Atheros L1 Gigabit Ethernet +device alc # Atheros AR8131/AR8132 Ethernet +device ale # Atheros AR8121/AR8113/AR8114 Ethernet +device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet +device bfe # Broadcom BCM440x 10/100 Ethernet +device bge # Broadcom BCM570xx Gigabit Ethernet +device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn +device dc # DEC/Intel 21143 and various workalikes +device et # Agere ET1310 10/100/Gigabit Ethernet +device fxp # Intel EtherExpress PRO/100B (82557, 82558) +device gem # Sun GEM/Sun ERI/Apple GMAC +device hme # Sun HME (Happy Meal Ethernet) +device jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet +device lge # Level 1 LXT1001 gigabit Ethernet +device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet +device nfe # nVidia nForce MCP on-board Ethernet +device nge # NatSemi DP83820 gigabit Ethernet +device pcn # AMD Am79C97x PCI 10/100 (precedence over 'le') +device re # RealTek 8139C+/8169/8169S/8110S +device rl # RealTek 8129/8139 +device sf # Adaptec AIC-6915 (``Starfire'') +device sge # Silicon Integrated Systems SiS190/191 +device sis # Silicon Integrated Systems SiS 900/SiS 7016 +device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet +device ste # Sundance ST201 (D-Link DFE-550TX) +device stge # Sundance/Tamarack TC9021 gigabit Ethernet +device tl # Texas Instruments ThunderLAN +device tx # SMC EtherPower II (83c170 ``EPIC'') +device vge # VIA VT612x gigabit Ethernet +device vr # VIA Rhine, Rhine II +device wb # Winbond W89C840F +device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') + +# Wireless NIC cards +device wlan # 802.11 support +options IEEE80211_DEBUG # enable debug msgs +options IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's +options IEEE80211_SUPPORT_MESH # enable 802.11s draft support +device wlan_wep # 802.11 WEP support +device wlan_ccmp # 802.11 CCMP support +device wlan_tkip # 802.11 TKIP support +device wlan_amrr # AMRR transmit rate control algorithm +device an # Aironet 4500/4800 802.11 wireless NICs. +device ath # Atheros NICs +device ath_pci # Atheros pci/cardbus glue +device ath_hal # pci/cardbus chip support +options AH_SUPPORT_AR5416 # enable AR5416 tx/rx descriptors +options AH_AR5416_INTERRUPT_MITIGATION # AR5416 interrupt mitigation +options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later +device ath_rate_sample # SampleRate tx rate control for ath +#device bwi # Broadcom BCM430x/BCM431x wireless NICs. +#device bwn # Broadcom BCM43xx wireless NICs. +device ipw # Intel 2100 wireless NICs. +device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. +device iwn # Intel 4965/1000/5000/6000 wireless NICs. +device malo # Marvell Libertas wireless NICs. +device mwl # Marvell 88W8363 802.11n wireless NICs. +device ral # Ralink Technology RT2500 wireless NICs. +device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. +device wpi # Intel 3945ABG wireless NICs. + +# Pseudo devices. +device loop # Network loopback +device random # Entropy device +device padlock_rng # VIA Padlock RNG +device rdrand_rng # Intel Bull Mountain RNG +device ether # Ethernet support +device vlan # 802.1Q VLAN support +device tun # Packet tunnel. +device md # Memory "disks" +device gif # IPv6 and IPv4 tunneling +device firmware # firmware assist module + +# The `bpf' device enables the Berkeley Packet Filter. +# Be aware of the administrative consequences of enabling this! +# Note that 'bpf' is required for DHCP. +device bpf # Berkeley packet filter + +# USB support +options USB_DEBUG # enable debug msgs +device uhci # UHCI PCI->USB interface +device ohci # OHCI PCI->USB interface +device ehci # EHCI PCI->USB interface (USB 2.0) +device xhci # XHCI PCI->USB interface (USB 3.0) +device usb # USB Bus (required) +device ukbd # Keyboard +device umass # Disks/Mass storage - Requires scbus and da + +# Sound support +device sound # Generic sound driver (required) +device snd_cmi # CMedia CMI8338/CMI8738 +device snd_csa # Crystal Semiconductor CS461x/428x +device snd_emu10kx # Creative SoundBlaster Live! and Audigy +device snd_es137x # Ensoniq AudioPCI ES137x +device snd_hda # Intel High Definition Audio +device snd_ich # Intel, NVidia and other ICH AC'97 Audio +device snd_via8233 # VIA VT8233x Audio + +# MMC/SD +device mmc # MMC/SD bus +device mmcsd # MMC/SD memory card +device sdhci # Generic PCI SD Host Controller + +# VirtIO support +device virtio # Generic VirtIO bus (required) +device virtio_pci # VirtIO PCI device +device vtnet # VirtIO Ethernet device +device virtio_blk # VirtIO Block device +device virtio_scsi # VirtIO SCSI device +device virtio_balloon # VirtIO Memory Balloon device + +# HyperV drivers and enchancement support +# NOTE: HYPERV depends on hyperv. They must be added or removed together. +options HYPERV # Hyper-V kernel infrastructure +device hyperv # HyperV drivers + +# Xen HVM Guest Optimizations +# NOTE: XENHVM depends on xenpci. They must be added or removed together. +options XENHVM # Xen HVM kernel infrastructure +device xenpci # Xen HVM Hypervisor services driver + +# VMware support +device vmx # VMware VMXNET3 Ethernet + +# Netmap provides direct access to TX/RX rings on supported NICs +device netmap # netmap(4) support + -- 2.37.1 From 7d84d85da3cbb935c5fdcd7caabc6a3016433175 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2015 17:30:20 +0200 Subject: [PATCH 009/310] ELECTRO_BLOAT: ElectroBSDify the copy Mainly by removing stuff that depends on proprietary software ElectroBSD doesn't ship with or stuff that isn't relevant for ElectroBSD. While at it, import the 'device crypto' line from GENERIC. While we don't need it for IPSEC, having it in the kernel makes upgrading from FreeBSD to ElectroBSD more convenient when using cloudiatr. By default cloudiatr only adds currenty-loaded modules to the bpool, so if crypto.ko is part of the currently running kernel, but the installed kernel requires it as module, the newly-setup system will not boot unless the user adds the module manually. Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 76 +++++++++++------------------------- 1 file changed, 22 insertions(+), 54 deletions(-) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index 3d2cc39c6353..a61f4e5eee3a 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -1,25 +1,25 @@ -# -# GENERIC -- Generic kernel configuration file for FreeBSD/amd64 -# -# For more information on this file, please read the config(5) manual page, -# and/or the handbook section on Kernel Configuration Files: -# -# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html -# -# The handbook is also available locally in /usr/share/doc/handbook -# if you've installed the doc distribution, otherwise always see the -# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the -# latest information. -# -# An exhaustive list of options and more detailed explanations of the -# device lines is also present in the ../../conf/NOTES and NOTES files. -# If you are in doubt as to the purpose or necessity of a line, check first -# in NOTES. -# -# $FreeBSD$ +# ELECTRO_BLOAT -- Modified copy of the GENERIC kernel configuration file +# Used for the release media. +ident ELECTRO_BLOAT + +# One of ElectroBSD's most important features: +# hacker-movie-compatible colors by default! +options SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK) + +# Add HTTP accept filter support. The "performance gains" might +# be dubious, but adding it results in nicer logs for applications +# that use it (because requests are less intangled). +options ACCEPT_FILTER_HTTP + +# Build pf into the kernel. It doesn't hurt and supposedly +# works around various bugs that only affect the module build. +device pf + +############################################################################## +# Everything below comes from GENERIC, but "offending" lines have been removed +############################################################################## cpu HAMMER -ident GENERIC makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support @@ -29,7 +29,6 @@ options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols options TCP_OFFLOAD # TCP offload -options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists @@ -84,10 +83,6 @@ options KDB_TRACE # Print a stack trace for a panic. options DDB # Support DDB. options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver -options INVARIANTS # Enable calls of extra sanity checking -options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS -options WITNESS # Enable checks to detect deadlocks and cycles -options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones # Make an SMP-capable kernel by default @@ -102,9 +97,6 @@ options ACPI_DMAR device pci options PCI_IOV # PCI SR-IOV support -# Floppy drives -device fdc - # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers @@ -121,13 +113,9 @@ options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series -device isp # Qlogic family -#device ispfw # Firmware for QLogic HBAs- normally a module device mpt # LSI-Logic MPT-Fusion device mps # LSI-Logic MPT-Fusion 2 device mpr # LSI-Logic MPT-Fusion 3 -#device ncr # NCR/Symbios Logic -device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') device trm # Tekram DC395U/UW/F DC315U adapters device adv # Advansys SCSI adapters @@ -151,10 +139,6 @@ device amr # AMI MegaRAID device arcmsr # Areca SATA II RAID device ciss # Compaq Smart RAID 5* device dpt # DPT Smartcache III, IV - See NOTES for options -device hptmv # Highpoint RocketRAID 182x -device hptnr # Highpoint DC7280, R750 -device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx -device hpt27xx # Highpoint RocketRAID 27xx device iir # Intel Integrated RAID device ips # IBM (Adaptec) ServeRAID device mly # Mylex AcceleRAID/eXtremeRAID @@ -167,10 +151,7 @@ device aacp # SCSI passthrough for aac (requires CAM) device aacraid # Adaptec by PMC RAID device ida # Compaq Smart RAID device mfi # LSI MegaRAID SAS -device mlx # Mylex DAC960 family device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s -#XXX pointer/int warnings -#device pst # Promise Supertrak SX6000 device twe # 3ware ATA RAID # NVM Express (NVMe) support @@ -219,7 +200,6 @@ device ppi # Parallel port interface device device puc # Multi I/O cards and multi-channel UARTs # PCI Ethernet NICs. -device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel PRO/1000 Gigabit Ethernet Family device igb # Intel PRO/1000 PCIE Server Gigabit Family @@ -239,7 +219,6 @@ device ae # Attansic/Atheros L2 FastEthernet device age # Attansic/Atheros L1 Gigabit Ethernet device alc # Atheros AR8131/AR8132 Ethernet device ale # Atheros AR8121/AR8113/AR8114 Ethernet -device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet device bfe # Broadcom BCM440x 10/100 Ethernet device bge # Broadcom BCM570xx Gigabit Ethernet device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn @@ -288,14 +267,7 @@ options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later device ath_rate_sample # SampleRate tx rate control for ath #device bwi # Broadcom BCM430x/BCM431x wireless NICs. #device bwn # Broadcom BCM43xx wireless NICs. -device ipw # Intel 2100 wireless NICs. -device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. -device iwn # Intel 4965/1000/5000/6000 wireless NICs. device malo # Marvell Libertas wireless NICs. -device mwl # Marvell 88W8363 802.11n wireless NICs. -device ral # Ralink Technology RT2500 wireless NICs. -device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. -device wpi # Intel 3945ABG wireless NICs. # Pseudo devices. device loop # Network loopback @@ -327,7 +299,6 @@ device umass # Disks/Mass storage - Requires scbus and da # Sound support device sound # Generic sound driver (required) device snd_cmi # CMedia CMI8338/CMI8738 -device snd_csa # Crystal Semiconductor CS461x/428x device snd_emu10kx # Creative SoundBlaster Live! and Audigy device snd_es137x # Ensoniq AudioPCI ES137x device snd_hda # Intel High Definition Audio @@ -348,8 +319,6 @@ device virtio_scsi # VirtIO SCSI device device virtio_balloon # VirtIO Memory Balloon device # HyperV drivers and enchancement support -# NOTE: HYPERV depends on hyperv. They must be added or removed together. -options HYPERV # Hyper-V kernel infrastructure device hyperv # HyperV drivers # Xen HVM Guest Optimizations @@ -360,6 +329,5 @@ device xenpci # Xen HVM Hypervisor services driver # VMware support device vmx # VMware VMXNET3 Ethernet -# Netmap provides direct access to TX/RX rings on supported NICs -device netmap # netmap(4) support - +# The crypto framework is required by IPSEC +device crypto # Required by IPSEC -- 2.37.1 From 7e9efe3167fb4d003c3e5b4b99620e2fb209cc85 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 27 Mar 2013 11:28:27 +0100 Subject: [PATCH 010/310] Import ELECTRO_BEER Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BEER | 257 ++++++++++++++++++++++++++++++++++++ 1 file changed, 257 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BEER diff --git a/sys/amd64/conf/ELECTRO_BEER b/sys/amd64/conf/ELECTRO_BEER new file mode 100644 index 000000000000..3fb26ab1675f --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BEER @@ -0,0 +1,257 @@ +include ELECTRO_BLOAT + +ident ELECTRO_BEER + +# Debugging for use in -current +nooptions INVARIANTS # Enable calls of extra sanity checking +nooptions INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +nooptions WITNESS # Enable checks to detect deadlocks and cycles +nooptions WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +nooptions FLOWTABLE # per-cpu routing cache +nooptions XENHVM # Include Xen support + +nooptions SCTP # Stream Control Transmission Protocol + +nodevice fdc + +nodevice ataraid # ATA RAID drives +nodevice atapifd # ATAPI floppy drives +nodevice atapist # ATAPI tape drives + +# ATA controllers +nodevice mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA +nodevice siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA + +# SCSI Controllers +nodevice ahc # AHA2940 and onboard AIC7xxx devices +nooptions AHC_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~128k to driver. +nodevice amd # AMD 53C974 (Tekram DC-390(T)) +nodevice esp # AMD Am53C974 (Tekram DC-390(T)) +nodevice hptiop # Highpoint RocketRaid 3xxx series +nodevice isp # Qlogic family +nodevice ispfw # Firmware for QLogic HBAs- normally a module +nodevice mpt # LSI-Logic MPT-Fusion +nodevice mps # LSI-Logic MPT-Fusion 2 +nodevice ncr # NCR/Symbios Logic +nodevice sym # NCR/Symbios Logic (newer chipsets + those of `ncr') +nodevice trm # Tekram DC395U/UW/F DC315U adapters + +nodevice adv # Advansys SCSI adapters +nodevice adw # Advansys wide SCSI adapters +nodevice aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. +nodevice bt # Buslogic/Mylex MultiMaster SCSI adapters + +# RAID controllers interfaced to the SCSI subsystem +nodevice amr # AMI MegaRAID +nodevice arcmsr # Areca SATA II RAID +#XXX it is not 64-bit clean, -scottl +nodevice asr # DPT SmartRAID V, VI and Adaptec SCSI RAID +nodevice ciss # Compaq Smart RAID 5* +nodevice dpt # DPT Smartcache III, IV - See NOTES for options +nodevice hptmv # Highpoint RocketRAID 182x +nodevice hptnr # Highpoint DC7280, R750 +nodevice hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx +nodevice hpt27xx # Highpoint RocketRAID 27xx + +nodevice iir # Intel Integrated RAID +nodevice ips # IBM (Adaptec) ServeRAID +nodevice mly # Mylex AcceleRAID/eXtremeRAID +nodevice twa # 3ware 9000 series PATA/SATA RAID +nodevice tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller + +# RAID controllers +nodevice aac # Adaptec FSA RAID +nodevice aacraid # Adaptec by PMC RAID +nodevice aacp # SCSI passthrough for aac (requires CAM) +nodevice ida # Compaq Smart RAID +nodevice mfi # LSI MegaRAID SAS +nodevice mlx # Mylex DAC960 family +#XXX pointer/int warnings +nodevice pst # Promise Supertrak SX6000 +nodevice twe # 3ware ATA RAID + + +# Parallel port +nodevice ppc +nodevice ppbus # Parallel port bus (required) +nodevice lpt # Printer +nodevice plip # TCP/IP over parallel +nodevice ppi # Parallel port interface device +nodevice vpo # Requires scbus and da + +# If you've got a "dumb" serial or parallel PCI card that is +# supported by the puc(4) glue driver, uncomment the following +# line to enable it (connects to sio, uart and/or ppc drivers): +nodevice puc + +# PCI Ethernet NICs. +nodevice bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE +nodevice de # DEC/Intel DC21x4x (``Tulip'') +nodevice em # Intel PRO/1000 Gigabit Ethernet Family +nodevice igb # Intel PRO/1000 PCIE Server Gigabit Family +nodevice ixgbe # Intel PRO/10GbE PCIE Ethernet Family +nodevice le # AMD Am7900 LANCE and Am79C9xx PCnet +nodevice ti # Alteon Networks Tigon I/II gigabit Ethernet +nodevice txp # 3Com 3cR990 (``Typhoon'') +nodevice vx # 3Com 3c590, 3c595 (``Vortex'') + +# PCI Ethernet NICs that use the common MII bus controller code. +# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! + +nodevice ae # Attansic/Atheros L2 FastEthernet +nodevice age # Attansic/Atheros L1 Gigabit Ethernet +nodevice alc # Atheros AR8131/AR8132 Ethernet +nodevice ale # Atheros AR8121/AR8113/AR8114 Ethernet +nodevice bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet +nodevice bfe # Broadcom BCM440x 10/100 Ethernet +nodevice cas # Sun Cassini/Cassini+ and NS DP83065 Saturn +nodevice dc # DEC/Intel 21143 and various workalikes +nodevice et # Agere ET1310 10/100/Gigabit Ethernet +nodevice fxp # Intel EtherExpress PRO/100B (82557, 82558) +nodevice gem # Sun GEM/Sun ERI/Apple GMAC +nodevice hme # Sun HME (Happy Meal Ethernet) +nodevice jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet +nodevice lge # Level 1 LXT1001 gigabit Ethernet +nodevice msk # Marvell/SysKonnect Yukon II Gigabit Ethernet +nodevice nfe # nVidia nForce MCP on-board Ethernet +nodevice nge # NatSemi DP83820 gigabit Ethernet +nodevice nve # nVidia nForce MCP on-board Ethernet Networking +nodevice pcn # AMD Am79C97x PCI 10/100 (precedence over 'le') +nodevice re # RealTek 8139C+/8169/8169S/8110S +nodevice rl # RealTek 8129/8139 +nodevice sf # Adaptec AIC-6915 (``Starfire'') +nodevice sge # Silicon Integrated Systems SiS190/191 +nodevice sis # Silicon Integrated Systems SiS 900/SiS 7016 +nodevice sk # SysKonnect SK-984x & SK-982x gigabit Ethernet +nodevice ste # Sundance ST201 (D-Link DFE-550TX) +nodevice stge # Sundance/Tamarack TC9021 gigabit Ethernet +nodevice tl # Texas Instruments ThunderLAN +nodevice tx # SMC EtherPower II (83c170 ``EPIC'') +nodevice vge # VIA VT612x gigabit Ethernet +nodevice vr # VIA Rhine, Rhine II +nodevice wb # Winbond W89C840F +nodevice xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') + +# ISA Ethernet NICs. pccard NICs included. +nodevice cs # Crystal Semiconductor CS89x0 NIC +# 'device ed' requires 'device miibus' +nodevice ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards +nodevice ex # Intel EtherExpress Pro/10 and Pro/10+ +nodevice ep # Etherlink III based cards +nodevice fe # Fujitsu MB8696x based cards +nodevice sn # SMC's 9000 series of Ethernet chips +nodevice xe # Xircom pccard Ethernet + +nodevice zyd # ZyDAS zb1211/zb1211b wireless NICs +nodevice urio # Diamond Rio 500 MP3 player + +# Wireless NIC cards +nodevice an # Aironet 4500/4800 802.11 wireless NICs. +nodevice ath # Atheros NIC's +nodevice ath_pci # Atheros pci/cardbus glue +nodevice ath_hal # pci/cardbus chip support +nodevice ath_rate_sample # SampleRate tx rate control for ath +nodevice bwi # Broadcom BCM430x/BCM431x wireless NICs. +nodevice bwn # Broadcom BCM43xx wireless NICs. +nodevice ipw # Intel 2100 wireless NICs. +nodevice iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. +nodevice iwn # Intel 4965/1000/5000/6000 wireless NICs. +nodevice malo # Marvell Libertas wireless NICs. +nodevice mwl # Marvell 88W8363 802.11n wireless NICs. +nodevice ral # Ralink Technology RT2500 wireless NICs. +nodevice wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. +nodevice wpi # Intel 3945ABG wireless NICs. + +# Pseudo devices. +nodevice padlock_rng # VIA Padlock RNG +nodevice rdrand_rng # Intel Bull Mountain RNG +nodevice gif # IPv6 and IPv4 tunneling +nodevice faith # IPv6-to-IPv4 relaying (translation) + +# USB Serial devices +nodevice uark # Technologies ARK3116 based serial adapters +nodevice ubsa # Belkin F5U103 and compatible serial adapters +nodevice uftdi # For FTDI usb serial adapters +nodevice uipaq # Some WinCE based devices +nodevice uplcom # Prolific PL-2303 serial adapters +nodevice uslcom # SI Labs CP2101/CP2102 serial adapters +nodevice uvisor # Visor and Palm devices +nodevice uvscom # USB serial support for DDI pocket's PHS +# USB Ethernet, requires miibus +nodevice aue # ADMtek USB Ethernet +nodevice axe # ASIX Electronics USB Ethernet +nodevice cdce # Generic USB over Ethernet +nodevice cue # CATC USB Ethernet +nodevice kue # Kawasaki LSI USB Ethernet +nodevice rue # RealTek RTL8150 USB Ethernet +nodevice udav # Davicom DM9601E USB + +# USB support + +nodevice uhci # UHCI PCI->USB interface +nodevice ohci # OHCI PCI->USB interface +nodevice ehci # EHCI PCI->USB interface (USB 2.0) +nodevice xhci # XHCI PCI->USB interface (USB 3.0) +nodevice usb # USB Bus (required) +nodevice ukbd # Keyboard +nodevice umass # Disks/Mass storage - Requires scbus and da + +# FireWire support +nodevice firewire # FireWire bus code +nodevice sbp # SCSI over FireWire (Requires scbus and da) +nodevice fwe # Ethernet over FireWire (non-standard!) +nodevice fwip # IP over FireWire (RFC 2734,3146) +nodevice dcons # Dumb console driver +nodevice dcons_crom # Configuration ROM for dcons + +# Sound support +nodevice snd_es137x # Ensoniq AudioPCI ES137x +nodevice snd_ich # Intel, NVidia and other ICH AC'97 Audio +nodevice snd_uaudio # USB Audio +nodevice snd_via8233 # VIA VT8233x Audio +nodevice snd_cmi # CMedia CMI8338/CMI8738 +nodevice snd_csa # Crystal Semiconductor CS461x/428x +nodevice snd_emu10kx # Creative SoundBlaster Live! and Audigy + +# VirtIO support +nodevice virtio # Generic VirtIO bus (required) +nodevice virtio_pci # VirtIO PCI device +nodevice vtnet # VirtIO Ethernet device +nodevice virtio_blk # VirtIO Block device +nodevice virtio_scsi # VirtIO SCSI device +nodevice virtio_balloon # VirtIO Memory Balloon device + +# HyperV drivers +nodevice hyperv # HyperV drivers + +# Xen support +nodevice xenpci # Generic Xen bus + +# VMware support +nodevice vmx # VMware VMXNET3 Ethernet + +# Same for Intel processors +device coretemp + +# man 4 cpuctl +device cpuctl # CPU control pseudo-device + +# UTF-8 in console (8.x+) +options TEKEN_UTF8 + +nodevice netmap + +# Disabling them prevents ZFS from being loaded. +# Should be bisected. +#nooptions NFS_ROOT +#nooptions NFSLOCKD +#nooptions NFSD +#nooptions NFSCL +nodevice ahd +nodevice mpr +nodevice isci +nodevice ses +nodevice mrsas +nodevice ixl +nodevice ixlv -- 2.37.1 From 2874361075f7ce62d8f12e834d131a654b9830d1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 14 Apr 2015 17:43:38 +0200 Subject: [PATCH 011/310] Add ELECTRO_BEER for i386 Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BEER | 1 + 1 file changed, 1 insertion(+) create mode 120000 sys/i386/conf/ELECTRO_BEER diff --git a/sys/i386/conf/ELECTRO_BEER b/sys/i386/conf/ELECTRO_BEER new file mode 120000 index 000000000000..4a483bb3a8ea --- /dev/null +++ b/sys/i386/conf/ELECTRO_BEER @@ -0,0 +1 @@ +../../amd64/conf/ELECTRO_BEER \ No newline at end of file -- 2.37.1 From 44c5a63e80f9e5e0452805c1de0a99673b9e98c5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 28 Jun 2015 17:06:35 +0200 Subject: [PATCH 012/310] i386: Copy GENERIC to ELECTRO_BLOAT Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BLOAT | 382 ++++++++++++++++++++++++++++++++++++ 1 file changed, 382 insertions(+) create mode 100644 sys/i386/conf/ELECTRO_BLOAT diff --git a/sys/i386/conf/ELECTRO_BLOAT b/sys/i386/conf/ELECTRO_BLOAT new file mode 100644 index 000000000000..aefc50753f0d --- /dev/null +++ b/sys/i386/conf/ELECTRO_BLOAT @@ -0,0 +1,382 @@ +# +# GENERIC -- Generic kernel configuration file for FreeBSD/i386 +# +# For more information on this file, please read the config(5) manual page, +# and/or the handbook section on Kernel Configuration Files: +# +# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html +# +# The handbook is also available locally in /usr/share/doc/handbook +# if you've installed the doc distribution, otherwise always see the +# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the +# latest information. +# +# An exhaustive list of options and more detailed explanations of the +# device lines is also present in the ../../conf/NOTES and NOTES files. +# If you are in doubt as to the purpose or necessity of a line, check first +# in NOTES. +# +# $FreeBSD$ + +cpu I486_CPU +cpu I586_CPU +cpu I686_CPU +ident GENERIC + +makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols +makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support + +options SCHED_ULE # ULE scheduler +options PREEMPTION # Enable kernel thread preemption +options INET # InterNETworking +options INET6 # IPv6 communications protocols +options IPSEC # IP (v4/v6) security +options TCP_OFFLOAD # TCP offload +options SCTP # Stream Control Transmission Protocol +options FFS # Berkeley Fast Filesystem +options SOFTUPDATES # Enable FFS soft updates support +options UFS_ACL # Support for access control lists +options UFS_DIRHASH # Improve performance on big directories +options UFS_GJOURNAL # Enable gjournal-based UFS journaling +options QUOTA # Enable disk quotas for UFS +options MD_ROOT # MD is a potential root device +options NFSCL # Network Filesystem Client +options NFSD # Network Filesystem Server +options NFSLOCKD # Network Lock Manager +options NFS_ROOT # NFS usable as /, requires NFSCL +options MSDOSFS # MSDOS Filesystem +options CD9660 # ISO 9660 Filesystem +options PROCFS # Process filesystem (requires PSEUDOFS) +options PSEUDOFS # Pseudo-filesystem framework +options GEOM_PART_GPT # GUID Partition Tables. +options GEOM_RAID # Soft RAID functionality. +options GEOM_LABEL # Provides labelization +options COMPAT_FREEBSD4 # Compatible with FreeBSD4 +options COMPAT_FREEBSD5 # Compatible with FreeBSD5 +options COMPAT_FREEBSD6 # Compatible with FreeBSD6 +options COMPAT_FREEBSD7 # Compatible with FreeBSD7 +options COMPAT_FREEBSD9 # Compatible with FreeBSD9 +options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI +options KTRACE # ktrace(1) support +options STACK # stack(9) support +options SYSVSHM # SYSV-style shared memory +options SYSVMSG # SYSV-style message queues +options SYSVSEM # SYSV-style semaphores +options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions +options PRINTF_BUFR_SIZE=128 # Prevent printf output being interspersed. +options KBD_INSTALL_CDEV # install a CDEV entry in /dev +options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(4) +options AUDIT # Security event auditing +options CAPABILITY_MODE # Capsicum capability mode +options CAPABILITIES # Capsicum capabilities +options MAC # TrustedBSD MAC Framework +options KDTRACE_HOOKS # Kernel DTrace hooks +options DDB_CTF # Kernel ELF linker loads CTF data +options INCLUDE_CONFIG_FILE # Include this file in kernel +options RACCT # Resource accounting framework +options RACCT_DEFAULT_TO_DISABLED # Set kern.racct.enable=0 by default +options RCTL # Resource limits + +# Debugging support. Always need this: +options KDB # Enable kernel debugger support. +options KDB_TRACE # Print a stack trace for a panic. +# For full debugger support use (turn off in stable branch): +options DDB # Support DDB. +options GDB # Support remote GDB. +options DEADLKRES # Enable the deadlock resolver +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones + +# To make an SMP kernel, the next two lines are needed +options SMP # Symmetric MultiProcessor Kernel +device apic # I/O APIC + +# CPU frequency control +device cpufreq + +# Bus support. +device acpi +device pci +options PCI_HP # PCI-Express native HotPlug +options PCI_IOV # PCI SR-IOV support + +# Floppy drives +device fdc + +# ATA controllers +device ahci # AHCI-compatible SATA controllers +device ata # Legacy ATA/SATA controllers +device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA +device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA + +# SCSI Controllers +device ahc # AHA2940 and onboard AIC7xxx devices +options AHC_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~128k to driver. +device ahd # AHA39320/29320 and onboard AIC79xx devices +options AHD_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~215k to driver. +device esp # AMD Am53C974 (Tekram DC-390(T)) +device hptiop # Highpoint RocketRaid 3xxx series +device isp # Qlogic family +#device ispfw # Firmware for QLogic HBAs- normally a module +device mpt # LSI-Logic MPT-Fusion +device mps # LSI-Logic MPT-Fusion 2 +device mpr # LSI-Logic MPT-Fusion 3 +#device ncr # NCR/Symbios Logic +device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') +device trm # Tekram DC395U/UW/F DC315U adapters + +device adv # Advansys SCSI adapters +device adw # Advansys wide SCSI adapters +device aha # Adaptec 154x SCSI adapters +device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. +device bt # Buslogic/Mylex MultiMaster SCSI adapters + +device ncv # NCR 53C500 +device nsp # Workbit Ninja SCSI-3 +device stg # TMC 18C30/18C50 +device isci # Intel C600 SAS controller + +# ATA/SCSI peripherals +device scbus # SCSI bus (required for ATA/SCSI) +device ch # SCSI media changers +device da # Direct Access (disks) +device sa # Sequential Access (tape etc) +device cd # CD +device pass # Passthrough device (direct ATA/SCSI access) +device ses # Enclosure Services (SES and SAF-TE) +#device ctl # CAM Target Layer + +# RAID controllers interfaced to the SCSI subsystem +device amr # AMI MegaRAID +device arcmsr # Areca SATA II RAID +device ciss # Compaq Smart RAID 5* +device dpt # DPT Smartcache III, IV - See NOTES for options +device hptmv # Highpoint RocketRAID 182x +device hptnr # Highpoint DC7280, R750 +device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx +device hpt27xx # Highpoint RocketRAID 27xx +device iir # Intel Integrated RAID +device ips # IBM (Adaptec) ServeRAID +device mly # Mylex AcceleRAID/eXtremeRAID +device twa # 3ware 9000 series PATA/SATA RAID +device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller + +# RAID controllers +device aac # Adaptec FSA RAID +device aacp # SCSI passthrough for aac (requires CAM) +device aacraid # Adaptec by PMC RAID +device ida # Compaq Smart RAID +device mfi # LSI MegaRAID SAS +device mlx # Mylex DAC960 family +device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s +device pmspcv # PMC-Sierra SAS/SATA Controller driver +device pst # Promise Supertrak SX6000 +device twe # 3ware ATA RAID + +# NVM Express (NVMe) support +device nvme # base NVMe driver +device nvd # expose NVMe namespace as disks, depends on nvme + +# atkbdc0 controls both the keyboard and the PS/2 mouse +device atkbdc # AT keyboard controller +device atkbd # AT keyboard +device psm # PS/2 mouse + +device kbdmux # keyboard multiplexer + +device vga # VGA video card driver +options VESA # Add support for VESA BIOS Extensions (VBE) + +device splash # Splash screen and screen saver support + +# syscons is the default console driver, resembling an SCO console +device sc +options SC_PIXEL_MODE # add support for the raster text mode + +# vt is the new video console driver +device vt +device vt_vga + +device agp # support several AGP chipsets + +# Power management support (see NOTES for more options) +#device apm +# Add suspend/resume support for the i8254. +device pmtimer + +# PCCARD (PCMCIA) support +# PCMCIA and cardbus bridge support +device cbb # cardbus (yenta) bridge +device pccard # PC Card (16-bit) bus +device cardbus # CardBus (32-bit) bus + +# Serial (COM) ports +device uart # Generic UART driver + +# Parallel port +device ppc +device ppbus # Parallel port bus (required) +device lpt # Printer +device ppi # Parallel port interface device +#device vpo # Requires scbus and da + +device puc # Multi I/O cards and multi-channel UARTs + +# PCI Ethernet NICs. +device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE +device de # DEC/Intel DC21x4x (``Tulip'') +device em # Intel PRO/1000 Gigabit Ethernet Family +device igb # Intel PRO/1000 PCIE Server Gigabit Family +device ixgb # Intel PRO/10GbE Ethernet Card +device le # AMD Am7900 LANCE and Am79C9xx PCnet +device ti # Alteon Networks Tigon I/II gigabit Ethernet +device txp # 3Com 3cR990 (``Typhoon'') +device vx # 3Com 3c590, 3c595 (``Vortex'') + +# PCI Ethernet NICs that use the common MII bus controller code. +# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! +device miibus # MII bus support +device ae # Attansic/Atheros L2 FastEthernet +device age # Attansic/Atheros L1 Gigabit Ethernet +device alc # Atheros AR8131/AR8132 Ethernet +device ale # Atheros AR8121/AR8113/AR8114 Ethernet +device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet +device bfe # Broadcom BCM440x 10/100 Ethernet +device bge # Broadcom BCM570xx Gigabit Ethernet +device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn +device dc # DEC/Intel 21143 and various workalikes +device et # Agere ET1310 10/100/Gigabit Ethernet +device fxp # Intel EtherExpress PRO/100B (82557, 82558) +device gem # Sun GEM/Sun ERI/Apple GMAC +device hme # Sun HME (Happy Meal Ethernet) +device jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet +device lge # Level 1 LXT1001 gigabit Ethernet +device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet +device nfe # nVidia nForce MCP on-board Ethernet +device nge # NatSemi DP83820 gigabit Ethernet +device pcn # AMD Am79C97x PCI 10/100 (precedence over 'le') +device re # RealTek 8139C+/8169/8169S/8110S +device rl # RealTek 8129/8139 +device sf # Adaptec AIC-6915 (``Starfire'') +device sge # Silicon Integrated Systems SiS190/191 +device sis # Silicon Integrated Systems SiS 900/SiS 7016 +device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet +device ste # Sundance ST201 (D-Link DFE-550TX) +device stge # Sundance/Tamarack TC9021 gigabit Ethernet +device tl # Texas Instruments ThunderLAN +device tx # SMC EtherPower II (83c170 ``EPIC'') +device vge # VIA VT612x gigabit Ethernet +device vr # VIA Rhine, Rhine II +device vte # DM&P Vortex86 RDC R6040 Fast Ethernet +device wb # Winbond W89C840F +device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') + +# ISA Ethernet NICs. pccard NICs included. +device cs # Crystal Semiconductor CS89x0 NIC +# 'device ed' requires 'device miibus' +device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards +device ex # Intel EtherExpress Pro/10 and Pro/10+ +device ep # Etherlink III based cards +device fe # Fujitsu MB8696x based cards +device ie # EtherExpress 8/16, 3C507, StarLAN 10 etc. +device sn # SMC's 9000 series of Ethernet chips +device xe # Xircom pccard Ethernet + +# Wireless NIC cards +device wlan # 802.11 support +options IEEE80211_DEBUG # enable debug msgs +options IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's +options IEEE80211_SUPPORT_MESH # enable 802.11s draft support +device wlan_wep # 802.11 WEP support +device wlan_ccmp # 802.11 CCMP support +device wlan_tkip # 802.11 TKIP support +device wlan_amrr # AMRR transmit rate control algorithm +device an # Aironet 4500/4800 802.11 wireless NICs. +device ath # Atheros NICs +device ath_pci # Atheros pci/cardbus glue +device ath_hal # pci/cardbus chip support +options AH_SUPPORT_AR5416 # enable AR5416 tx/rx descriptors +options AH_AR5416_INTERRUPT_MITIGATION # AR5416 interrupt mitigation +options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later +device ath_rate_sample # SampleRate tx rate control for ath +#device bwi # Broadcom BCM430x/BCM431x wireless NICs. +#device bwn # Broadcom BCM43xx wireless NICs. +device ipw # Intel 2100 wireless NICs. +device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. +device iwn # Intel 4965/1000/5000/6000 wireless NICs. +device malo # Marvell Libertas wireless NICs. +device mwl # Marvell 88W8363 802.11n wireless NICs. +device ral # Ralink Technology RT2500 wireless NICs. +device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. +#device wl # Older non 802.11 Wavelan wireless NIC. +device wpi # Intel 3945ABG wireless NICs. + +# Pseudo devices. +device loop # Network loopback +device random # Entropy device +device padlock_rng # VIA Padlock RNG +device rdrand_rng # Intel Bull Mountain RNG +device ether # Ethernet support +device vlan # 802.1Q VLAN support +device tun # Packet tunnel. +device md # Memory "disks" +device gif # IPv6 and IPv4 tunneling +device firmware # firmware assist module + +# The `bpf' device enables the Berkeley Packet Filter. +# Be aware of the administrative consequences of enabling this! +# Note that 'bpf' is required for DHCP. +device bpf # Berkeley packet filter + +# USB support +options USB_DEBUG # enable debug msgs +device uhci # UHCI PCI->USB interface +device ohci # OHCI PCI->USB interface +device ehci # EHCI PCI->USB interface (USB 2.0) +device xhci # XHCI PCI->USB interface (USB 3.0) +device usb # USB Bus (required) +device ukbd # Keyboard +device umass # Disks/Mass storage - Requires scbus and da + +# Sound support +device sound # Generic sound driver (required) +device snd_cmi # CMedia CMI8338/CMI8738 +device snd_csa # Crystal Semiconductor CS461x/428x +device snd_emu10kx # Creative SoundBlaster Live! and Audigy +device snd_es137x # Ensoniq AudioPCI ES137x +device snd_hda # Intel High Definition Audio +device snd_ich # Intel, NVidia and other ICH AC'97 Audio +device snd_via8233 # VIA VT8233x Audio + +# MMC/SD +device mmc # MMC/SD bus +device mmcsd # MMC/SD memory card +device sdhci # Generic PCI SD Host Controller + +# VirtIO support +device virtio # Generic VirtIO bus (required) +device virtio_pci # VirtIO PCI device +device vtnet # VirtIO Ethernet device +device virtio_blk # VirtIO Block device +device virtio_scsi # VirtIO SCSI device +device virtio_balloon # VirtIO Memory Balloon device + +# HyperV drivers and enchancement support +device hyperv # HyperV drivers + +# Xen HVM Guest Optimizations +# NOTE: XENHVM depends on xenpci. They must be added or removed together. +options XENHVM # Xen HVM kernel infrastructure +device xenpci # Xen HVM Hypervisor services driver + +# VMware support +device vmx # VMware VMXNET3 Ethernet + +# The crypto framework is required by IPSEC +device crypto # Required by IPSEC -- 2.37.1 From 7bc33b4ff5718c9b789121e9fb3f318bdd333d7a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 28 Jun 2015 17:55:42 +0200 Subject: [PATCH 013/310] Adjust shiny new ELECTRO_BLOAT i386 for ElectroBSD Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BLOAT | 81 ++++++++----------------------------- 1 file changed, 17 insertions(+), 64 deletions(-) diff --git a/sys/i386/conf/ELECTRO_BLOAT b/sys/i386/conf/ELECTRO_BLOAT index aefc50753f0d..fcfb7c314411 100644 --- a/sys/i386/conf/ELECTRO_BLOAT +++ b/sys/i386/conf/ELECTRO_BLOAT @@ -1,27 +1,11 @@ -# -# GENERIC -- Generic kernel configuration file for FreeBSD/i386 -# -# For more information on this file, please read the config(5) manual page, -# and/or the handbook section on Kernel Configuration Files: -# -# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html -# -# The handbook is also available locally in /usr/share/doc/handbook -# if you've installed the doc distribution, otherwise always see the -# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the -# latest information. -# -# An exhaustive list of options and more detailed explanations of the -# device lines is also present in the ../../conf/NOTES and NOTES files. -# If you are in doubt as to the purpose or necessity of a line, check first -# in NOTES. -# -# $FreeBSD$ - -cpu I486_CPU -cpu I586_CPU +# ELECTRO_BLOAT -- Modified copy of the GENERIC kernel configuration file +# Used for the release media. + cpu I686_CPU -ident GENERIC +ident ELECTRO_BLOAT + +# Hacker-movie-compatible colors +options SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK) makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support @@ -30,9 +14,7 @@ options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols -options IPSEC # IP (v4/v6) security options TCP_OFFLOAD # TCP offload -options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists @@ -49,12 +31,7 @@ options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. -options GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization -options COMPAT_FREEBSD4 # Compatible with FreeBSD4 -options COMPAT_FREEBSD5 # Compatible with FreeBSD5 -options COMPAT_FREEBSD6 # Compatible with FreeBSD6 -options COMPAT_FREEBSD7 # Compatible with FreeBSD7 options COMPAT_FREEBSD9 # Compatible with FreeBSD9 options COMPAT_FREEBSD10 # Compatible with FreeBSD10 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI @@ -85,10 +62,6 @@ options KDB_TRACE # Print a stack trace for a panic. options DDB # Support DDB. options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver -options INVARIANTS # Enable calls of extra sanity checking -options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS -options WITNESS # Enable checks to detect deadlocks and cycles -options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones # To make an SMP kernel, the next two lines are needed @@ -101,15 +74,12 @@ device cpufreq # Bus support. device acpi device pci -options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support -# Floppy drives -device fdc - # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers +options ATA_STATIC_ID # Static device numbering device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA @@ -123,12 +93,9 @@ options AHD_REG_PRETTY_PRINT # Print register bitfields in debug device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series device isp # Qlogic family -#device ispfw # Firmware for QLogic HBAs- normally a module device mpt # LSI-Logic MPT-Fusion device mps # LSI-Logic MPT-Fusion 2 device mpr # LSI-Logic MPT-Fusion 3 -#device ncr # NCR/Symbios Logic -device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') device trm # Tekram DC395U/UW/F DC315U adapters device adv # Advansys SCSI adapters @@ -157,10 +124,6 @@ device amr # AMI MegaRAID device arcmsr # Areca SATA II RAID device ciss # Compaq Smart RAID 5* device dpt # DPT Smartcache III, IV - See NOTES for options -device hptmv # Highpoint RocketRAID 182x -device hptnr # Highpoint DC7280, R750 -device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx -device hpt27xx # Highpoint RocketRAID 27xx device iir # Intel Integrated RAID device ips # IBM (Adaptec) ServeRAID device mly # Mylex AcceleRAID/eXtremeRAID @@ -173,16 +136,10 @@ device aacp # SCSI passthrough for aac (requires CAM) device aacraid # Adaptec by PMC RAID device ida # Compaq Smart RAID device mfi # LSI MegaRAID SAS -device mlx # Mylex DAC960 family device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s -device pmspcv # PMC-Sierra SAS/SATA Controller driver device pst # Promise Supertrak SX6000 device twe # 3ware ATA RAID -# NVM Express (NVMe) support -device nvme # base NVMe driver -device nvd # expose NVMe namespace as disks, depends on nvme - # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard @@ -205,8 +162,6 @@ device vt_vga device agp # support several AGP chipsets -# Power management support (see NOTES for more options) -#device apm # Add suspend/resume support for the i8254. device pmtimer @@ -229,7 +184,6 @@ device ppi # Parallel port interface device device puc # Multi I/O cards and multi-channel UARTs # PCI Ethernet NICs. -device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel PRO/1000 Gigabit Ethernet Family device igb # Intel PRO/1000 PCIE Server Gigabit Family @@ -246,7 +200,6 @@ device ae # Attansic/Atheros L2 FastEthernet device age # Attansic/Atheros L1 Gigabit Ethernet device alc # Atheros AR8131/AR8132 Ethernet device ale # Atheros AR8121/AR8113/AR8114 Ethernet -device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet device bfe # Broadcom BCM440x 10/100 Ethernet device bge # Broadcom BCM570xx Gigabit Ethernet device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn @@ -307,15 +260,7 @@ options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later device ath_rate_sample # SampleRate tx rate control for ath #device bwi # Broadcom BCM430x/BCM431x wireless NICs. #device bwn # Broadcom BCM43xx wireless NICs. -device ipw # Intel 2100 wireless NICs. -device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. -device iwn # Intel 4965/1000/5000/6000 wireless NICs. device malo # Marvell Libertas wireless NICs. -device mwl # Marvell 88W8363 802.11n wireless NICs. -device ral # Ralink Technology RT2500 wireless NICs. -device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. -#device wl # Older non 802.11 Wavelan wireless NIC. -device wpi # Intel 3945ABG wireless NICs. # Pseudo devices. device loop # Network loopback @@ -347,7 +292,6 @@ device umass # Disks/Mass storage - Requires scbus and da # Sound support device sound # Generic sound driver (required) device snd_cmi # CMedia CMI8338/CMI8738 -device snd_csa # Crystal Semiconductor CS461x/428x device snd_emu10kx # Creative SoundBlaster Live! and Audigy device snd_es137x # Ensoniq AudioPCI ES137x device snd_hda # Intel High Definition Audio @@ -380,3 +324,12 @@ device vmx # VMware VMXNET3 Ethernet # The crypto framework is required by IPSEC device crypto # Required by IPSEC + +# This used to be required for ZFS when compiled with clang. +# For details see UPDATING entry 20121223. After r286288 it's +# probably no longer necessary, but for now we keep it anyway. +options KSTACK_PAGES=4 + +# Increase the size of the kernel virtual address space +# so ZFS can cache more stuff. +options KVA_PAGES=512 -- 2.37.1 From 94058e17cd77dec75c2f0c80a5670d7ea7f7ae13 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2015 17:38:09 +0200 Subject: [PATCH 014/310] Change amd64 default KERNCONF to ELECTRO_BLOAT Obtained from: ElectroBSD --- Makefile.inc1 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Makefile.inc1 b/Makefile.inc1 index 3b63c6f85ffc..98917d809185 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1620,6 +1620,8 @@ KERNCONF?= GENERIC64 KERNCONF?= GENERIC64LE .elif ${TARGET_ARCH} == "powerpcspe" KERNCONF?= MPC85XXSPE +.elif ${TARGET_ARCH} == "amd64" +KERNCONF?= ELECTRO_BLOAT .else KERNCONF?= GENERIC .endif -- 2.37.1 From d2e2308ad9628a1b82e881fb075dcf639c31b19c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 22 May 2016 12:10:03 +0200 Subject: [PATCH 015/310] Use ELECTRO_BLOAT as default on i386 Obtained from: ElectroBSD --- Makefile.inc1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.inc1 b/Makefile.inc1 index 98917d809185..b080f8a0b464 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1620,7 +1620,7 @@ KERNCONF?= GENERIC64 KERNCONF?= GENERIC64LE .elif ${TARGET_ARCH} == "powerpcspe" KERNCONF?= MPC85XXSPE -.elif ${TARGET_ARCH} == "amd64" +.elif ${TARGET_ARCH} == "amd64" || ${TARGET_ARCH} == "i386" KERNCONF?= ELECTRO_BLOAT .else KERNCONF?= GENERIC -- 2.37.1 From 9866d38a680a75b2b2fd12db8d2bee40b697c366 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Aug 2011 19:19:51 +0200 Subject: [PATCH 016/310] Register the product id for Feiya Memory Bar Obtained from: ElectroBSD --- sys/dev/usb/usbdevs | 1 + 1 file changed, 1 insertion(+) diff --git a/sys/dev/usb/usbdevs b/sys/dev/usb/usbdevs index 032c6be1fbfe..e2ba529bfc56 100644 --- a/sys/dev/usb/usbdevs +++ b/sys/dev/usb/usbdevs @@ -1979,6 +1979,7 @@ product FALCOM SAMBA 0x0005 FTDI compatible adapter /* FEIYA products */ product FEIYA DUMMY 0x0000 Dummy product +product FEIYA MEMORY_BAR 0x1000 Memory Bar product FEIYA 5IN1 0x1132 5-in-1 Card Reader product FEIYA ELANGO 0x6200 MicroSDHC Card Reader product FEIYA AC110 0x6300 AC-110 Card Reader -- 2.37.1 From 6d7a07d982eb02975f703ef5c1e00b7d59a7b9c4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Aug 2011 19:22:46 +0200 Subject: [PATCH 017/310] Add the full name of 'Feya Technology Corp.' Obtained from: ElectroBSD --- sys/dev/usb/usbdevs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/usb/usbdevs b/sys/dev/usb/usbdevs index e2ba529bfc56..eeb6c82f5e62 100644 --- a/sys/dev/usb/usbdevs +++ b/sys/dev/usb/usbdevs @@ -455,7 +455,7 @@ vendor DIGIANSWER 0x08fd Digianswer vendor AUTHENTEC 0x08ff AuthenTec vendor AUDIOTECHNICA 0x0909 Audio-Technica vendor TRUMPION 0x090a Trumpion Microelectronics -vendor FEIYA 0x090c Feiya +vendor FEIYA 0x090c Feiya Technology Corp. vendor ALATION 0x0910 Alation Systems vendor GLOBESPAN 0x0915 Globespan vendor CONCORDCAMERA 0x0919 Concord Camera -- 2.37.1 From 5d13ae3a768ea926c4966b9977e489ec8f49be7f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 20 Dec 2013 18:45:00 +0100 Subject: [PATCH 018/310] Let g_eli_keyfiles_load() log the size of the keyfile loaded Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 2493fd31c7de..529e0f006224 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -1120,8 +1120,8 @@ g_eli_keyfiles_load(struct hmac_ctx *ctx, const char *provider) name); return (0); } - G_ELI_DEBUG(1, "Loaded keyfile %s for %s (type: %s).", file, - provider, name); + G_ELI_DEBUG(1, "Loaded keyfile %s for %s (type: %s) (size: %d).", file, + provider, name, (unsigned)size); g_eli_crypto_hmac_update(ctx, data, size); } } -- 2.37.1 From a1ebf79d5c70f8144ceed87d588e9128405d32a0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 7 Jan 2015 18:50:18 +0100 Subject: [PATCH 019/310] bge(4): Default to disallowing ASF It causes watchdog timeouts and undiagnosed permanent unresponsivenes on at least the 'CHIP ID 0x05784100; ASIC REV 0x5784; CHIP REV 0x57841;' in the evo-iv08 DL120 G6 I'm using for testing. I don't have time to debug the underlying cause right now and users who actually want ASF and have systems where it works can always enable it through loader.conf. Obtained from: ElectroBSD --- share/man/man4/bge.4 | 2 +- sys/dev/bge/if_bge.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/share/man/man4/bge.4 b/share/man/man4/bge.4 index 50c0cf9c9b8c..c09b35edc088 100644 --- a/share/man/man4/bge.4 +++ b/share/man/man4/bge.4 @@ -200,7 +200,7 @@ prompt before booting the kernel, or stored in .It Va hw.bge.allow_asf Allow the ASF feature for cooperating with IPMI. Can cause system lockup problems on a small number of systems. -Enabled by default. +Disabled by default. .It Va dev.bge.%d.msi Non-zero value enables MSI support on the Ethernet hardware. The default value is 1. diff --git a/sys/dev/bge/if_bge.c b/sys/dev/bge/if_bge.c index e63a821888e4..0f5142b91e85 100644 --- a/sys/dev/bge/if_bge.c +++ b/sys/dev/bge/if_bge.c @@ -539,7 +539,7 @@ MODULE_PNP_INFO("U16:vendor;U16:device", pci, bge, bge_devs, nitems(bge_devs) - 1); DRIVER_MODULE(miibus, bge, miibus_driver, miibus_devclass, 0, 0); -static int bge_allow_asf = 1; +static int bge_allow_asf = 0; static SYSCTL_NODE(_hw, OID_AUTO, bge, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, "BGE driver parameters"); -- 2.37.1 From 006eb413dc3ba654c4e5dbb3b7145fb1d0964d70 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 7 Jan 2015 21:25:47 +0100 Subject: [PATCH 020/310] Assign random IP id values by default so users don't have to clown around with the sysctl themselves Obtained from: ElectroBSD --- sys/netinet/ip_id.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/netinet/ip_id.c b/sys/netinet/ip_id.c index b52f5403b332..11191357e96d 100644 --- a/sys/netinet/ip_id.c +++ b/sys/netinet/ip_id.c @@ -138,8 +138,8 @@ static void ipid_sysuninit(void); SYSCTL_DECL(_net_inet_ip); SYSCTL_PROC(_net_inet_ip, OID_AUTO, random_id, CTLTYPE_INT | CTLFLAG_VNET | CTLFLAG_RW | CTLFLAG_MPSAFE, - &VNET_NAME(ip_do_randomid), 0, sysctl_ip_randomid, "IU", - "Assign random ip_id values"); + &VNET_NAME(ip_do_randomid), 1, sysctl_ip_randomid, "IU", + "Assign random ip_id values. Important for Tor relays and a good idea in general."); SYSCTL_INT(_net_inet_ip, OID_AUTO, rfc6864, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_rfc6864), 0, "Use constant IP ID for atomic datagrams"); -- 2.37.1 From d19b0acbdb58e30ccf7e79e4d596b999f2da76c5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 11 Feb 2015 12:19:24 +0100 Subject: [PATCH 021/310] Add vendor copyright ... after putting on my robe and "police educational technican" hat. Obviously this commit is optional. Feel free to import any other ElectroBSD commit without including this one. Obtained from: ElectroBSD --- sys/sys/copyright.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/sys/copyright.h b/sys/sys/copyright.h index c9e070859e6c..388e123b5702 100644 --- a/sys/sys/copyright.h +++ b/sys/sys/copyright.h @@ -29,7 +29,7 @@ /* Add a FreeBSD vendor copyright here */ #define COPYRIGHT_Vendor \ - "" + "Copyright (c) 2010-2022 Fabian Keil - IT-Beratung und Polizei-Erziehung\n" /* FreeBSD */ #define COPYRIGHT_FreeBSD \ -- 2.37.1 From 2703c791eb11c363f95f56d18d66c3fe87561e22 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 Jan 2016 13:45:32 +0100 Subject: [PATCH 022/310] mdocml: Change OS name used in man page headers to ElectroBSD Obtained from: ElectroBSD --- contrib/mandoc/msec.in | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/contrib/mandoc/msec.in b/contrib/mandoc/msec.in index fc198202fbf1..b76a433865f4 100644 --- a/contrib/mandoc/msec.in +++ b/contrib/mandoc/msec.in @@ -22,14 +22,14 @@ * Be sure to escape strings. */ -LINE("1", "FreeBSD General Commands Manual") -LINE("2", "FreeBSD System Calls Manual") -LINE("3", "FreeBSD Library Functions Manual") +LINE("1", "ElectroBSD General Commands Manual") +LINE("2", "ElectroBSD System Calls Manual") +LINE("3", "ElectroBSD Library Functions Manual") LINE("3lua", "Lua Library Functions Manual") LINE("3p", "Perl Library Functions Manual") -LINE("4", "FreeBSD Kernel Interfaces Manual") -LINE("5", "FreeBSD File Formats Manual") -LINE("6", "FreeBSD Games Manual") -LINE("7", "FreeBSD Miscellaneous Information Manual") -LINE("8", "FreeBSD System Manager\'s Manual") -LINE("9", "FreeBSD Kernel Developer\'s Manual") +LINE("4", "ElectroBSD Kernel Interfaces Manual") +LINE("5", "ElectroBSD File Formats Manual") +LINE("6", "ElectroBSD Games Manual") +LINE("7", "ElectroBSD Miscellaneous Information Manual") +LINE("8", "ElectroBSD System Manager\'s Manual") +LINE("9", "ElectroBSD Kernel Developer\'s Manual") -- 2.37.1 From d3905f1f432576fcf2457b68f95e05b88d985f92 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 24 Jan 2016 20:24:00 +0100 Subject: [PATCH 023/310] clang: Set CLANG_VENDOR to ElectroBSD Obtained from: ElectroBSD --- lib/clang/include/clang/Basic/Version.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/clang/include/clang/Basic/Version.inc b/lib/clang/include/clang/Basic/Version.inc index d0c3b2db52c3..776f8fb0047c 100644 --- a/lib/clang/include/clang/Basic/Version.inc +++ b/lib/clang/include/clang/Basic/Version.inc @@ -6,4 +6,4 @@ #define CLANG_VERSION_MINOR 0 #define CLANG_VERSION_PATCHLEVEL 5 -#define CLANG_VENDOR "FreeBSD " +#define CLANG_VENDOR "ElectroBSD " -- 2.37.1 From 7aa0c3cfd5e97e1f4bbbf684d2371d8ced2d5cf9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 24 Feb 2015 19:35:03 +0100 Subject: [PATCH 024/310] Let rc.d/motd work with unames other than FreeBSD Obtained from: ElectroBSD --- libexec/rc/rc.d/motd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libexec/rc/rc.d/motd b/libexec/rc/rc.d/motd index e63973945f9d..05ffc2be47fe 100755 --- a/libexec/rc/rc.d/motd +++ b/libexec/rc/rc.d/motd @@ -46,7 +46,7 @@ motd_start() T=`mktemp -t motd` uname -v | sed -e 's,^\([^#]*\) #\(.* [1-2][0-9][0-9][0-9]\).*/\([^\]*\) $,\1 (\3) #\2,' > ${T} - cat "${TEMPLATE}" >> ${T} + awk '{if (NR == 1) {if ($1 == "'"$(uname)"'") {next} else {print "\n"$0}} else {print}}' < "${TEMPLATE}" >> ${T} install -C -o root -g wheel -m "${PERMS}" "$T" "${TARGET}" rm -f "$T" -- 2.37.1 From 03b2f8424725a2dd79cc84f8c33da09f4b45989b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 4 May 2015 18:48:20 +0200 Subject: [PATCH 025/310] g_multipath: Add sysctl to disable tasting Obtained from: ElectroBSD --- sys/geom/multipath/g_multipath.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sys/geom/multipath/g_multipath.c b/sys/geom/multipath/g_multipath.c index a721b0bc4459..e31a7ebcbfac 100644 --- a/sys/geom/multipath/g_multipath.c +++ b/sys/geom/multipath/g_multipath.c @@ -62,6 +62,9 @@ SYSCTL_UINT(_kern_geom_multipath, OID_AUTO, debug, CTLFLAG_RW, static u_int g_multipath_exclusive = 1; SYSCTL_UINT(_kern_geom_multipath, OID_AUTO, exclusive, CTLFLAG_RW, &g_multipath_exclusive, 0, "Exclusively open providers"); +static u_int g_multipath_enable_tasting = 1; +SYSCTL_UINT(_kern_geom_multipath, OID_AUTO, taste, CTLFLAG_RW, + &g_multipath_enable_tasting, 0, "Enable multipath tasting. May cause conflicts."); SDT_PROVIDER_DECLARE(geom); SDT_PROBE_DEFINE2(geom, multipath, config, restore, "char*", "char*"); @@ -824,6 +827,9 @@ g_multipath_taste(struct g_class *mp, struct g_provider *pp, int flags __unused) g_topology_assert(); + if (g_multipath_enable_tasting == 0) + return (NULL); + gp = g_new_geomf(mp, "multipath:taste"); gp->start = g_multipath_start; gp->access = g_multipath_access; -- 2.37.1 From d7c40e0f14e741922b49154a7015b3dd601a7192 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 9 May 2015 14:32:31 +0200 Subject: [PATCH 026/310] motd.template: Customize for ElectroBSD Obtained from: ElectroBSD --- usr.bin/login/motd.template | 20 ++++---------------- 1 file changed, 4 insertions(+), 16 deletions(-) diff --git a/usr.bin/login/motd.template b/usr.bin/login/motd.template index ea98a3266c3a..98f2cd24d2f4 100644 --- a/usr.bin/login/motd.template +++ b/usr.bin/login/motd.template @@ -1,20 +1,8 @@ +ElectroBSD ?.?.? (UNKNOWN) -Welcome to FreeBSD! +Welcome to ElectroBSD! -Release Notes, Errata: https://www.FreeBSD.org/releases/ -Security Advisories: https://www.FreeBSD.org/security/ -FreeBSD Handbook: https://www.FreeBSD.org/handbook/ -FreeBSD FAQ: https://www.FreeBSD.org/faq/ -Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/ -FreeBSD Forums: https://forums.FreeBSD.org/ - -Documents installed with the system are in the /usr/local/share/doc/freebsd/ -directory, or can be installed later with: pkg install en-freebsd-doc -For other languages, replace "en" with a language code like de or fr. - -Show the version of FreeBSD installed: freebsd-version ; uname -a -Please include that output and any error messages when posting questions. -Introduction to manual pages: man man -FreeBSD directory layout: man hier +For details see: +https://www.ElectroBSD.org/ To change this login announcement, see motd(5). -- 2.37.1 From 5a66a48737d16e5ec53353b17ceffff81fc7db2d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 May 2015 14:18:43 +0200 Subject: [PATCH 027/310] release: Load usb modules through loader.conf so 'memstick' works with ELECTRO_BEER Obtained from: ElectroBSD --- release/Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/release/Makefile b/release/Makefile index 1288f750441a..71e0e83c34db 100644 --- a/release/Makefile +++ b/release/Makefile @@ -213,6 +213,9 @@ dvd: packagesystem echo hostid_enable=\"NO\" >> ${.TARGET}/etc/rc.conf echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf + for module in usb usb_quirk ehci umass; do \ + echo $${module}_load=\"YES\" >> ${.TARGET}/boot/loader.conf + done cp ${.CURDIR}/rc.local ${.TARGET}/etc touch ${.TARGET} -- 2.37.1 From e6a3ab0a5172e6082d44b998fc2968312b5b9c0f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 May 2015 19:13:51 +0200 Subject: [PATCH 028/310] brand-fbsd.4th: Change OS name in banner to ElectroBSD While at it, suggest to resist unlawful police activities (German). Obtained from: ElectroBSD --- stand/forth/brand-fbsd.4th | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/stand/forth/brand-fbsd.4th b/stand/forth/brand-fbsd.4th index 84245ef232b6..6795df852abd 100644 --- a/stand/forth/brand-fbsd.4th +++ b/stand/forth/brand-fbsd.4th @@ -32,7 +32,7 @@ 1+ \ increase y for next time we're called ; -: brand ( x y -- ) \ "FreeBSD" [wide] logo in B/W (7 rows x 42 columns) +: brand ( x y -- ) \ "ElectroBSD" [wide] logo in B/W framebuffer? if s" term-putimage" sfind if @@ -46,13 +46,16 @@ then then - s" ______ ____ _____ _____ " brand+ - s" | ____| | _ \ / ____| __ \ " brand+ - s" | |___ _ __ ___ ___ | |_) | (___ | | | |" brand+ - s" | ___| '__/ _ \/ _ \| _ < \___ \| | | |" brand+ - s" | | | | | __/ __/| |_) |____) | |__| |" brand+ - s" | | | | | | || | | |" brand+ - s" |_| |_| \___|\___||____/|_____/|_____/ " brand+ + s" ______ _ _ ____ _____ _____" brand+ + s" | ____| | | | | _ \ / ____| __ \" brand+ + s" | |__ | | ___ ___| |_ _ __ ___ | |_) | (___ | | | |" brand+ + s" | __| | |/ _ \/ __| __| '__/ _ \| _ < \___ \| | | |" brand+ + s" | |____| | __/ (__| |_| | | (_) | |_) |____) | |__| |" brand+ + s" |______|_|\___|\___|\__|_| \___/|____/|_____/|_____/" brand+ + s" Polizei-Willkuer in Deinem Land? Das erfordert Widerstand!" brand+ + s" Zu Risiken und Nebenwirkungen fragen Sie Ihren Anwalt oder" brand+ + s" die Rote Hilfe." brand+ + 2drop ; -- 2.37.1 From 25238c72aba671a5771717238c4bca31944ca29f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 May 2015 19:25:56 +0200 Subject: [PATCH 029/310] beastie.4th: Think of the children and default to showing beastie instead of the sex toy Obtained from: ElectroBSD --- stand/forth/beastie.4th | 4 ++-- stand/forth/beastie.4th.8 | 5 ++--- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/stand/forth/beastie.4th b/stand/forth/beastie.4th index f64cf9f8720b..665b9abf2b7e 100644 --- a/stand/forth/beastie.4th +++ b/stand/forth/beastie.4th @@ -64,9 +64,9 @@ variable logoY s" loader_logo" getenv dup -1 = over 0= or if dup 0= if 2drop else drop then \ getenv result unused loader_color? if - s" try-include /boot/logo-orb.4th" + s" try-include /boot/logo-beastie.4th" else - s" try-include /boot/logo-orbbw.4th" + s" try-include /boot/logo-beastiebw.4th" then else 2drop ( c-addr/u -- ) \ getenv result unused diff --git a/stand/forth/beastie.4th.8 b/stand/forth/beastie.4th.8 index 97a5624d5a93..0d120d5a30f9 100644 --- a/stand/forth/beastie.4th.8 +++ b/stand/forth/beastie.4th.8 @@ -107,10 +107,9 @@ Selects the desired logo in the beastie boot menu. Possible values are: .Dq Li fbsdbw , .Dq Li beastie , -.Dq Li beastiebw , +.Dq Li beastiebw (default) , .Dq Li orb , -.Dq Li orbbw -(default), and +.Dq Li orbbw , and .Dq Li none . .It Va loader_logo_x Sets the desired column position of the logo. -- 2.37.1 From b298fa7acf1e11083baf3bbbc3f142a04d71f039 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 May 2015 19:39:56 +0200 Subject: [PATCH 030/310] menu.4th: Reduce visual noise by ditching the welcome text Obtained from: ElectroBSD --- stand/forth/menu.4th | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/forth/menu.4th b/stand/forth/menu.4th index 75e75e3e3654..ed14981670e2 100644 --- a/stand/forth/menu.4th +++ b/stand/forth/menu.4th @@ -470,7 +470,7 @@ also menu-infrastructure definitions \ Print the frame caption at (x,y) s" loader_menu_title" getenv dup -1 = if - drop s" Welcome to FreeBSD" + drop s" " then TRUE ( use default alignment ) s" loader_menu_title_align" getenv dup -1 <> if -- 2.37.1 From 68b1d713655927bcc0ca521b8bf06d47d92c5ca9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 May 2015 17:40:22 +0200 Subject: [PATCH 031/310] logo-beastiebw: Change the fork to a toilet brush Obtained from: ElectroBSD --- stand/forth/logo-beastiebw.4th | 40 +++++++++++++++++----------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/stand/forth/logo-beastiebw.4th b/stand/forth/logo-beastiebw.4th index 197099cda0bc..bce43f5b9d52 100644 --- a/stand/forth/logo-beastiebw.4th +++ b/stand/forth/logo-beastiebw.4th @@ -33,27 +33,27 @@ 1+ \ increase y for next time we're called ; -: logo ( x y -- ) \ B/W BSD mascot (19 rows x 34 columns) +: logo ( x y -- ) \ B/W BSD mascot with toilet brush - s" , ," logo+ - s" /( )`" logo+ - s" \ \___ / |" logo+ - s" /- _ `-/ '" logo+ - s" (/\/ \ \ /\" logo+ - s" / / | ` \" logo+ - s" O O ) / |" logo+ - s" `-^--'`< '" logo+ - s" (_.) _ ) /" logo+ - s" `.___/` /" logo+ - s" `-----' /" logo+ - s" <----. __ / __ \" logo+ - s" <----|====O)))==) \) /====|" logo+ - s" <----' `--' `.__,' \" logo+ - s" | |" logo+ - s" \ / /\" logo+ - s" ______( (_ / \______/" logo+ - s" ,' ,-----' |" logo+ - s" `--{__________)" logo+ + s" , ," logo+ + s" /( )`" logo+ + s" \ \___ / |" logo+ + s" /- _ `-/ '" logo+ + s" (/\/ \ \ /\" logo+ + s" / / | ` \" logo+ + s" O O ) / |" logo+ + s" `-^--'`< '" logo+ + s" (_.) _ ) /" logo+ + s" `.___/` /" logo+ + s" `-----' /" logo+ + s" ###### __ / __ \" logo+ + s" ######===O)))==) \) /====|" logo+ + s" ###### `--' `.__,' \" logo+ + s" | |" logo+ + s" \ / /\" logo+ + s" ______( (_ / \___/" logo+ + s" ,' ,-----' |" logo+ + s" `--{__________)" logo+ 2drop ; -- 2.37.1 From 2474ab3a9c617ec558aad5274f73ee96c6c3c672 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 May 2015 18:19:19 +0200 Subject: [PATCH 032/310] logo-beastie.4th: Replace fork with toilet brush Obtained from: ElectroBSD --- stand/forth/logo-beastie.4th | 40 ++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/stand/forth/logo-beastie.4th b/stand/forth/logo-beastie.4th index 671eb5e496b2..8441c9ddafbd 100644 --- a/stand/forth/logo-beastie.4th +++ b/stand/forth/logo-beastie.4th @@ -35,27 +35,27 @@ 1+ \ increase y for next time we're called ; -: logo ( x y -- ) \ color BSD mascot (19 rows x 34 columns) +: logo ( x y -- ) \ color BSD mascot with toilet brush - s" @[31m, ," logo+ - s" /( )`" logo+ - s" \ \___ / |" logo+ - s" /- @[m_@[31m `-/ '" logo+ - s" (@[m/\/ \@[31m \ /\" logo+ - s" @[m/ / |@[31m ` \" logo+ - s" @[34mO O @[m) @[31m/ |" logo+ - s" @[m`-^--'@[31m`< '" logo+ - s" (_.) _ ) /" logo+ - s" `.___/` /" logo+ - s" `-----' /" logo+ - s" @[33m<----.@[31m __ / __ \" logo+ - s" @[33m<----|====@[31mO)))@[33m==@[31m) \) /@[33m====|" logo+ - s" @[33m<----'@[31m `--' `.__,' \" logo+ - s" | |" logo+ - s" \ / /\" logo+ - s" @[36m______@[31m( (_ / \______/" logo+ - s" @[36m,' ,-----' |" logo+ - s" `--{__________)@[m" logo+ + s" @[31m, ," logo+ + s" /( )`" logo+ + s" \ \___ / |" logo+ + s" /- @[m_@[31m `-/ '" logo+ + s" (@[m/\/ \@[31m \ /\" logo+ + s" @[m/ / |@[31m ` \" logo+ + s" @[34mO O @[m) @[31m/ |" logo+ + s" @[m`-^--'@[31m`< '" logo+ + s" (_.) _ ) /" logo+ + s" `.___/` /" logo+ + s" `-----' /" logo+ + s" @[37m######@[31m __ / __ \" logo+ + s" @[37m######====@[31mO)))@[37m==@[31m) \) /@[37m====|" logo+ + s" @[37m######@[31m `--' `.__,' \" logo+ + s" | |" logo+ + s" \ / /\" logo+ + s" @[36m______@[31m( (_ / \_____/" logo+ + s" @[36m,' ,-----' |" logo+ + s" `--{__________)@[m" logo+ 2drop ; -- 2.37.1 From 4e13ec7ad7699531d2135d25b7ca4834baf92777 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 14 May 2015 10:57:01 +0200 Subject: [PATCH 033/310] rc.d/jail: Remove obnoxious warning about 'obsolete' jail_* variables They will not be removed from ElectroBSD until a replacement exists that is usable with shell scripts like ezjail without jumping through lots of hoops. Obtained from: ElectroBSD --- libexec/rc/rc.d/jail | 1 - 1 file changed, 1 deletion(-) diff --git a/libexec/rc/rc.d/jail b/libexec/rc/rc.d/jail index 1d544501e32b..a40080615be6 100755 --- a/libexec/rc/rc.d/jail +++ b/libexec/rc/rc.d/jail @@ -15,7 +15,6 @@ desc="Manage system jails" rcvar="jail_enable" start_cmd="jail_start" -start_postcmd="jail_warn" stop_cmd="jail_stop" config_cmd="jail_config" console_cmd="jail_console" -- 2.37.1 From 8b9d5278e79bd67f7349525544dde753c730948f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 14 May 2015 12:34:11 +0200 Subject: [PATCH 034/310] kern.opts.mk: Disable IPFILTER, IPSEC_SUPPORT, OFED, SOURCELESS_HOST and SOURCELESS_UCODE and enable REPRODUCIBLE_BUILDS (XXX) This commit should probably be split into three. src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- sys/conf/kern.opts.mk | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/sys/conf/kern.opts.mk b/sys/conf/kern.opts.mk index a6e0a48d3272..098152ee7c4e 100644 --- a/sys/conf/kern.opts.mk +++ b/sys/conf/kern.opts.mk @@ -36,17 +36,12 @@ __DEFAULT_YES_OPTIONS = \ FORMAT_EXTENSIONS \ INET \ INET6 \ - IPFILTER \ - IPSEC_SUPPORT \ ISCSI \ KERNEL_SYMBOLS \ NETGRAPH \ - OFED \ PF \ - SCTP_SUPPORT \ - SOURCELESS_HOST \ - SOURCELESS_UCODE \ SPLIT_KERNEL_DEBUG \ + REPRODUCIBLE_BUILD \ TESTS \ USB_GADGET_EXAMPLES \ ZFS @@ -57,8 +52,16 @@ __DEFAULT_NO_OPTIONS = \ INIT_ALL_PATTERN \ INIT_ALL_ZERO \ KERNEL_RETPOLINE \ + SCTP_SUPPORT \ RATELIMIT \ - REPRODUCIBLE_BUILD + IPFILTER \ + IPSEC_SUPPORT \ + NAND \ + OFED \ + RATELIMIT \ + SOURCELESS_HOST \ + SOURCELESS_UCODE + # Some options are totally broken on some architectures. We disable # them. If you need to enable them on an experimental basis, you -- 2.37.1 From 8dfd0bcb1927e3b3bdeeeb7d89750886f2260167 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 15 May 2015 13:11:57 +0200 Subject: [PATCH 035/310] boot/newvers.sh branding: Change bootprog_name[] to ElectroBSD Obtained from: ElectroBSD --- stand/common/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/common/newvers.sh b/stand/common/newvers.sh index 714adba6c9cb..f5e7a04c40cf 100755 --- a/stand/common/newvers.sh +++ b/stand/common/newvers.sh @@ -50,7 +50,7 @@ u=${USER-root} h=${HOSTNAME-`hostname`} t=`date` #r=`head -n 6 $1 | tail -n 1 | awk -F: ' { print $1 } '` r=`awk -F: ' /^[0-9]\.[0-9]+:/ { print $1; exit }' $1` -bootprog_info="FreeBSD/${3} ${2}, Revision ${r}\\n" +bootprog_info="ElectroBSD/${3} ${2}, Revision ${r}\\n" if [ -n "${include_metadata}" ]; then bootprog_info="$bootprog_info(${t} ${u}@${h})\\n" fi -- 2.37.1 From 4195f3e48c274dedd39c43b4ed0a1c4afd1e2c9e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 18 May 2015 15:59:58 +0200 Subject: [PATCH 036/310] release: Default to not distributing the ports tree ... as there are too many license problems. Allow the user to set WITH_PORTS to distribute the ports tree anyway. Obtained from: ElectroBSD --- release/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/release/Makefile b/release/Makefile index 71e0e83c34db..f3de8c376d30 100644 --- a/release/Makefile +++ b/release/Makefile @@ -22,8 +22,8 @@ # PORTSDIR: location of ports tree to distribute (default: /usr/ports) # XTRADIR: xtra-bits-dir argument for /mkisoimages.sh # NOPKG: if set, do not distribute third-party packages -# NOPORTS: if set, do not distribute ports tree # NOSRC: if set, do not distribute source tree +# WITH_PORTS: if set, distribute ports tree provided it exists # WITH_DVD: if set, generate dvd1.iso # WITH_COMPRESSED_IMAGES: if set, compress installation images with xz(1) # (uncompressed images are not removed) @@ -71,7 +71,7 @@ VOLUME_LABEL= ${REVISION:C/[.-]/_/g}_${BRANCH:C/[.-]/_/g}_${TARGET_ARCH} VOLUME_LABEL= FreeBSD_Install .endif -.if !exists(${PORTSDIR}) +.if !exists(${PORTSDIR}) || !defined(WITH_PORTS) NOPORTS= true .endif -- 2.37.1 From d0f745f4f1cd7e170568155cfbad878e067d5661 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 May 2015 15:26:44 +0200 Subject: [PATCH 037/310] sys/boot/common/newvers.sh: Allow to overwrite the date to make boot loader binaries reproducible Obtained from: ElectroBSD --- stand/common/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/common/newvers.sh b/stand/common/newvers.sh index f5e7a04c40cf..e1d17e57d23c 100755 --- a/stand/common/newvers.sh +++ b/stand/common/newvers.sh @@ -46,7 +46,7 @@ done shift $((OPTIND - 1)) LC_ALL=C; export LC_ALL -u=${USER-root} h=${HOSTNAME-`hostname`} t=`date` +u=${USER-root} h=${HOSTNAME-`hostname`} t=${DATE-`date`} #r=`head -n 6 $1 | tail -n 1 | awk -F: ' { print $1 } '` r=`awk -F: ' /^[0-9]\.[0-9]+:/ { print $1; exit }' $1` -- 2.37.1 From c54abbcb4cbecf27a09481d00f9c68f726dad1ca Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 May 2015 15:36:48 +0200 Subject: [PATCH 038/310] sys/conf/newvers.sh: Allow to overwrite the build date embedded into the kernel This is a required step to get reproducible builds. Obtained from: ElectroBSD --- sys/conf/newvers.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index d9150ca83376..1c5ecf28f97a 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -180,7 +180,12 @@ v=${KERNEL_VERSION_NUMBER:-`cat version`} u=${USER:-root} d=$(pwd) h=${HOSTNAME:-$(hostname)} -if [ -n "$SOURCE_DATE_EPOCH" ]; then +if [ -n "${DATE}" ]; then + # SOURCE_DATE_EPOCH was added upstream in r291691 + # but the ElectroBSD build goo is still setting DATE + # which has a different format. + t=${DATE} +elif [ -n "$SOURCE_DATE_EPOCH" ]; then if ! t=$(date -r $SOURCE_DATE_EPOCH 2>/dev/null); then echo "Invalid SOURCE_DATE_EPOCH" >&2 exit 1 -- 2.37.1 From e20fff7f836baaaaa50f8f0a82768bb7c02087e0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 19 May 2015 16:06:01 +0200 Subject: [PATCH 039/310] release/amd64/make-memstick.sh: Use reproducible timestamps for the makefs image Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index c0e405c3a976..c13eb5aa71de 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -35,9 +35,11 @@ fi echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local -makefs -B little -o label=FreeBSD_Install -o version=2 ${2}.part ${1} +mtree -c -k time -p "${1}" | sed "s@time=.*@time=${EPOCH_DATE-0}.000000000@" > "${2}.mtree" +makefs -B little -o label=FreeBSD_Install -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local +rm "${2}.mtree" # Make an ESP in a file. espfilename=$(mktemp /tmp/efiboot.XXXXXX) -- 2.37.1 From c46761e7bcfe4ca40885dc8e05ef789d3aa4d7bf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 17 Sep 2015 11:53:45 +0200 Subject: [PATCH 040/310] make-memstick.sh: Additionally fake uid and gid on the created fs Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index c13eb5aa71de..4414912b138a 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -35,7 +35,12 @@ fi echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local -mtree -c -k time -p "${1}" | sed "s@time=.*@time=${EPOCH_DATE-0}.000000000@" > "${2}.mtree" +# Prepare mtree spec to fake timestamp, owner and group. +# As a result, man pages will be owned by root instead of man. +# Unfortunately we can't simply reuse ${1}/METALOG as it is incomplete. +mtree -c -k time -p "${1}" | sed \ + -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ + > "${2}.mtree" || return 1 makefs -B little -o label=FreeBSD_Install -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local -- 2.37.1 From 16bed850b84fa422ef6be84ae881175553674cc2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 30 Sep 2015 13:15:44 +0200 Subject: [PATCH 041/310] release/amd64/make-memstick.sh: Error out if mkimg fails instead of cleaning up Makes debugging more convenient. Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 4414912b138a..2bd21d7082af 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -56,6 +56,10 @@ mkimg -s mbr \ -p freebsd:-"mkimg -s bsd -b ${1}/boot/boot -p freebsd-ufs:=${2}.part" \ -a 2 \ -o ${2} +if [ $? -ne 0 ]; then + echo "mkimg failed" + exit 1 +fi rm ${espfilename} rm ${2}.part -- 2.37.1 From 27d37271d7383e99042c84abec4231c259071243 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 21 May 2015 18:40:56 +0200 Subject: [PATCH 042/310] release/amd64/make-memstick.sh: Allow to overwrite VOLUME_LABEL through the environment Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 2bd21d7082af..515b48824c8e 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -17,6 +17,7 @@ scriptdir=$(dirname $(realpath $0)) PATH=/bin:/usr/bin:/sbin:/usr/sbin export PATH +VOLUME_LABEL=${VOLUME_LABEL-"FreeBSD_Install"} if [ $# -ne 2 ]; then echo "make-memstick.sh /path/to/directory /path/to/image/file" @@ -33,7 +34,7 @@ if [ -e ${2} ]; then exit 1 fi -echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab +echo "/dev/ufs/${VOLUME_LABEL} / ufs ro,noatime 1 1" > ${1}/etc/fstab echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local # Prepare mtree spec to fake timestamp, owner and group. # As a result, man pages will be owned by root instead of man. @@ -41,7 +42,7 @@ echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local mtree -c -k time -p "${1}" | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 -makefs -B little -o label=FreeBSD_Install -o version=2 -F "${2}.mtree" ${2}.part ${1} +makefs -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local rm "${2}.mtree" -- 2.37.1 From 0cc03ecbab9931a02e12f1770d809501c9213d26 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 28 Jun 2015 13:45:09 +0200 Subject: [PATCH 043/310] Use make-memstick.sh's amd64 version on i386 as well Obtained from: ElectroBSD --- release/i386/make-memstick.sh | 45 +---------------------------------- 1 file changed, 1 insertion(+), 44 deletions(-) mode change 100755 => 120000 release/i386/make-memstick.sh diff --git a/release/i386/make-memstick.sh b/release/i386/make-memstick.sh deleted file mode 100755 index 6774e86ae550..000000000000 --- a/release/i386/make-memstick.sh +++ /dev/null @@ -1,44 +0,0 @@ -#!/bin/sh -# -# This script generates a "memstick image" (image that can be copied to a -# USB memory stick) from a directory tree. Note that the script does not -# clean up after itself very well for error conditions on purpose so the -# problem can be diagnosed (full filesystem most likely but ...). -# -# Usage: make-memstick.sh -# -# $FreeBSD$ -# - -set -e - -PATH=/bin:/usr/bin:/sbin:/usr/sbin -export PATH - -if [ $# -ne 2 ]; then - echo "make-memstick.sh /path/to/directory /path/to/image/file" - exit 1 -fi - -if [ ! -d ${1} ]; then - echo "${1} must be a directory" - exit 1 -fi - -if [ -e ${2} ]; then - echo "won't overwrite ${2}" - exit 1 -fi - -echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab -echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local -makefs -B little -o label=FreeBSD_Install -o version=2 ${2}.part ${1} -rm ${1}/etc/fstab -rm ${1}/etc/rc.conf.local - -mkimg -s mbr \ - -b ${1}/boot/mbr \ - -p freebsd:-"mkimg -s bsd -b ${1}/boot/boot -p freebsd-ufs:=${2}.part" \ - -o ${2} -rm ${2}.part - diff --git a/release/i386/make-memstick.sh b/release/i386/make-memstick.sh new file mode 120000 index 000000000000..978e04a02184 --- /dev/null +++ b/release/i386/make-memstick.sh @@ -0,0 +1 @@ +../amd64/make-memstick.sh \ No newline at end of file -- 2.37.1 From b435d60cbdb4a95fc35aaeecdd8f1c98067b3b71 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 2 Feb 2016 20:28:35 +0100 Subject: [PATCH 044/310] release/amd64/make-memstick.sh: Use the shiny new -T switch to clamp the remaining timestamps Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 515b48824c8e..9af4cff6e36a 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -42,7 +42,10 @@ echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local mtree -c -k time -p "${1}" | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 -makefs -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} +# The base makefs may not have -T support yet, +# thus we use the fresh one that goes into the image. +dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ + -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local rm "${2}.mtree" -- 2.37.1 From 1551e128b1cd035eb49c5f1f278e41c30c4a95a9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 21 May 2017 18:10:15 +0200 Subject: [PATCH 045/310] release/amd64/make-memstick.sh: Attempt to build the mtree spec reproducible Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 9af4cff6e36a..f227560da1d2 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -39,7 +39,7 @@ echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local # Prepare mtree spec to fake timestamp, owner and group. # As a result, man pages will be owned by root instead of man. # Unfortunately we can't simply reuse ${1}/METALOG as it is incomplete. -mtree -c -k time -p "${1}" | sed \ +mtree -c -k time -p "${1}" | mtree -C -k all | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 # The base makefs may not have -T support yet, -- 2.37.1 From 2620ba5b8b844a4ba4e1dbff7dee66c835b6576f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 21 May 2017 17:58:45 +0200 Subject: [PATCH 046/310] release/amd64/make-memstick.sh: Limit makefs to a single cpu ... to see if it's sufficient to get reproducible images without having to limit the number of cores for everything. Spoiler alert: Nope. Currently the image checksum is the same when using 1, 2 or 3 cores but changes when using 4: fk@t520 ~ $diffoscope-3.5 /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img --- /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img +++ /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img @@ -7641,16 +7641,16 @@ 000428b0 53 44 2d 31 31 2e 31 2d 50 52 45 52 45 4c 45 41 |SD-11.1-PRERELEA| 000428b0 53 44 2d 31 31 2e 31 2d 50 52 45 52 45 4c 45 41 |SD-11.1-PRERELEA| 000428c0 53 45 2d 61 6d 64 36 00 00 00 00 00 00 00 00 00 |SE-amd6.........| 000428c0 53 45 2d 61 6d 64 36 00 00 00 00 00 00 00 00 00 |SE-amd6.........| 000428d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000428d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * -00042940 00 75 92 01 08 00 00 00 00 79 92 01 08 00 00 00 |.u.......y......| -00042940 00 75 92 01 08 00 00 00 00 79 92 01 08 00 00 00 |.u.......y......| +00042940 00 85 92 01 08 00 00 00 00 89 92 01 08 00 00 00 |................| +00042940 00 85 92 01 08 00 00 00 00 89 92 01 08 00 00 00 |................| 00042950 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..| 00042950 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..| 00042960 00 00 00 00 00 00 00 00 d8 ec 08 00 00 00 00 00 |................| 00042960 00 00 00 00 00 00 00 00 d8 ec 08 00 00 00 00 00 |................| 00042970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00042970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * @@ -7724,16 +7724,16 @@ 000448b0 53 44 2d 31 31 2e 31 2d 50 52 45 52 45 4c 45 41 |SD-11.1-PRERELEA| 000448b0 53 44 2d 31 31 2e 31 2d 50 52 45 52 45 4c 45 41 |SD-11.1-PRERELEA| 000448c0 53 45 2d 61 6d 64 36 00 00 00 00 00 00 00 00 00 |SE-amd6.........| 000448c0 53 45 2d 61 6d 64 36 00 00 00 00 00 00 00 00 00 |SE-amd6.........| 000448d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000448d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * -00044940 00 75 92 01 08 00 00 00 00 79 92 01 08 00 00 00 |.u.......y......| -00044940 00 75 92 01 08 00 00 00 00 79 92 01 08 00 00 00 |.u.......y......| +00044940 00 85 92 01 08 00 00 00 00 89 92 01 08 00 00 00 |................| +00044940 00 85 92 01 08 00 00 00 00 89 92 01 08 00 00 00 |................| 00044950 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..| 00044950 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..| 00044960 00 00 00 00 00 00 00 00 d8 ec 08 00 00 00 00 00 |................| 00044960 00 00 00 00 00 00 00 00 d8 ec 08 00 00 00 00 00 |................| 00044970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00044970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * @@ -1048569,8 +1048569,8 @@ 00871140 84 1e 81 ff 50 38 0f 31 89 04 25 2c 84 1e 81 66 |....P8.1..%,...f| 00871140 84 1e 81 ff 50 38 0f 31 89 04 25 2c 84 1e 81 66 |....P8.1..%,...f| 00871150 c7 04 25 38 84 1e 81 08 01 c6 04 25 3b 84 1e 81 |..%8.......%;...| 00871150 c7 04 25 38 84 1e 81 08 01 c6 04 25 3b 84 1e 81 |..%8.......%;...| 00871160 00 8b 04 25 28 84 1e 81 8d 48 01 89 0c 25 28 84 |...%(....H...%(.| 00871160 00 8b 04 25 28 84 1e 81 8d 48 01 89 0c 25 28 84 |...%(....H...%(.| 00871170 1e 81 88 04 25 3a 84 1e 81 48 c7 c7 30 84 1e 81 |....%:...H..0...| -[ Too much input for diff (SHA1: 6efa7af7add1fe5ba288c5ea10dd9e2328a4c697) ] +[ Too much input for diff (SHA1: 867b19a5d3489d643cdeb48f1950d1d5f6af4f93) ] The changing bytes are in the UFS partition but apparently invisible from above: fk@t520 ~ $sudo /usr/src/release/scripts/image-checksum.sh -v /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img Mounting /dev/md0p2 at /mnt Running mtree, saving spec in /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img.mtree Unmounting /mnt ... Mounting /dev/md0p2 at /mnt Running ls to get inodes, saving output in /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img.ls Unmounting /mnt ... gpart checksum: df93d1be80c1870b38e2e9888c32898f090c72aad8f72838f8ee9117f15a326e Boot code checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 mtree checksum: 758bff96cef51a0fbc8f0aa8386a7c00ef084740d7c185a6d9152a1a9f17e3a3 ls checksum: 2c2d99a9409f9eb1b0d32b0defa65f5454017e29e57f7295c02c495c121f0dd0 Partial image checksum for /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img: dff88a8272549f8e01ad7cc325f5af284ba81f964bcc8001e373af287b5a4b05 Mounting /dev/md0p2 at /mnt Running mtree, saving spec in /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img.mtree Unmounting /mnt ... Mounting /dev/md0p2 at /mnt Running ls to get inodes, saving output in /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img.ls Unmounting /mnt ... gpart checksum: df93d1be80c1870b38e2e9888c32898f090c72aad8f72838f8ee9117f15a326e Boot code checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 mtree checksum: 758bff96cef51a0fbc8f0aa8386a7c00ef084740d7c185a6d9152a1a9f17e3a3 ls checksum: 2c2d99a9409f9eb1b0d32b0defa65f5454017e29e57f7295c02c495c121f0dd0 Partial image checksum for /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img: dff88a8272549f8e01ad7cc325f5af284ba81f964bcc8001e373af287b5a4b05 While this commit doesn't actually solve anything, it's kept for now as it doesn't hurt and it's useful to keep an example difference around. Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index f227560da1d2..32ee9eea7ec2 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -44,7 +44,10 @@ mtree -c -k time -p "${1}" | mtree -C -k all | sed \ > "${2}.mtree" || return 1 # The base makefs may not have -T support yet, # thus we use the fresh one that goes into the image. -dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ +# +# Bind makefs to a single cpu as the output is core dependant +# (XXX: doesn't seem to have the intended effect) +cpuset -l 0 dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local -- 2.37.1 From 8208db493b6e464ac84329c9c21230a17952b5bf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 May 2017 13:12:05 +0200 Subject: [PATCH 047/310] release/amd64/make-memstick.sh: Specify makefs debug flags which may or may not help Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 32ee9eea7ec2..ca8259a10354 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -42,12 +42,19 @@ echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local mtree -c -k time -p "${1}" | mtree -C -k all | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 + +#define DEBUG_FS_MAKEFS 0x00000400 +#define DEBUG_FS_CREATE_IMAGE 0x00001000 +#define DEBUG_FS_WRITE_FILE 0x00080000 +MAKEFS_DEBUG_FLAGS=0x81400 + # The base makefs may not have -T support yet, # thus we use the fresh one that goes into the image. # # Bind makefs to a single cpu as the output is core dependant # (XXX: doesn't seem to have the intended effect) cpuset -l 0 dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ + -d "${MAKEFS_DEBUG_FLAGS}" \ -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local -- 2.37.1 From 621f71e012e79740a93c687f75e0cc0076652f36 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 May 2015 18:45:24 +0200 Subject: [PATCH 048/310] release: Use a hack to recreate dist tarballs with reproducible timestamps (XXX: obsolete) Obtained from: ElectroBSD --- release/Makefile | 3 ++ release/scripts/tar-time-reset.sh | 64 +++++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+) create mode 100755 release/scripts/tar-time-reset.sh diff --git a/release/Makefile b/release/Makefile index f3de8c376d30..83837f01b1ce 100644 --- a/release/Makefile +++ b/release/Makefile @@ -238,6 +238,9 @@ mini-memstick.img: bootonly sh ${.CURDIR}/${TARGET}/make-memstick.sh bootonly ${.TARGET} packagesystem: base.txz kernel.txz ${EXTRA_PACKAGES} + for tarball in *.txz; do \ + sh ${.CURDIR}/scripts/tar-time-reset.sh $${tarball}; \ + done sh ${.CURDIR}/scripts/make-manifest.sh *.txz > MANIFEST touch ${.TARGET} diff --git a/release/scripts/tar-time-reset.sh b/release/scripts/tar-time-reset.sh new file mode 100755 index 000000000000..8186fd8d3e26 --- /dev/null +++ b/release/scripts/tar-time-reset.sh @@ -0,0 +1,64 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## +# +# This script resets the timestamps in a given tarfile to hopefully make +# it reproducible. This is a rather wasteful approach, but works for now. +# +# A better solution would be to patch bsdtar to optionally use a fixed +# time (without having to clown around with mtree specs). +# +########################################################################## + +main() { + local tarfile="${1}" \ + tempdir mtree_spec + + if [ $# -ne 1 ]; then + echo "$0 /path/to/tarfile" + exit 1 + fi + + if [ -z "${tarfile}" ]; then + echo "No tar file given" + return 1 + fi + tarfile="$(realpath "$tarfile")" + + tempdir=$(mktemp -d) || return 1 + mtree_spec=$(mktemp) || return 1 + + echo "Extracting tarfile ${tarfile}" + (cd "${tempdir}" && tar xvf "${tarfile}") || return 1 + + echo "Ditching original tarfile ${tarfile}" + rm "${tarfile}" + + echo "Creating mtree spec in ${mtree_spec}" + (cd "${tempdir}" && mtree -L -c -k time) | \ + sed "s@time=.*@time=${EPOCH_DATE-0}.000000000@" > "${mtree_spec}" + + echo "Creating tarfile ${tarfile}" + (cd "${tempdir}" && tar acLvf "${tarfile}" @"${mtree_spec}") || return 1 + + echo "Ditching ${tempdir}" + rm -r "${tempdir}" || return 1 + echo "Ditching ${mtree_spec}" + rm "${mtree_spec}" || return 1 +} + +main "${@}" -- 2.37.1 From 1bfd5b7dd07e10f0479f175304c8968de09926c9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 22 May 2015 11:33:50 +0200 Subject: [PATCH 049/310] release/scripts/mm-mtree.sh: Allow to build the mtree spec for mergemaster reproducible This relies on NetBSD mtree which has been the default for a while now. Obtained from: ElectroBSD --- release/scripts/mm-mtree.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/release/scripts/mm-mtree.sh b/release/scripts/mm-mtree.sh index 790154595798..8c87bae03ab3 100755 --- a/release/scripts/mm-mtree.sh +++ b/release/scripts/mm-mtree.sh @@ -140,8 +140,12 @@ find ${TEMPROOT} -type f -size 0 -delete 2>/dev/null find -d ${TEMPROOT} -type d -empty -delete 2>/dev/null # Build the mtree database in a temporary location. +# The second mtree call is used to get a reproducible result +# without embedded hostname, user name and creation timestamp. +# XXX: Might be obsolete after r301584. MTREENEW=`mktemp -t mergemaster.mtree` -mtree -nci -p ${TEMPROOT} -k size,md5digest > ${MTREENEW} 2>/dev/null +mtree -mci -p ${TEMPROOT} -k size,md5digest 2>/dev/null | \ + mtree -C -k all > ${MTREENEW} if [ -s "${MTREENEW}" ]; then echo "*** Saving mtree database for future upgrades" -- 2.37.1 From 4891e621831a92bda33a9cbad0aa6b945e2dca04 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 18 May 2015 19:17:14 +0200 Subject: [PATCH 050/310] release/Makefile: Don't create matroshka src tarballs that contain other tarballs ... if DESTDIR isn't set to a reasonable value. While at it, exclude *.orig files as well. Obtained from: ElectroBSD --- release/Makefile | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/release/Makefile b/release/Makefile index 83837f01b1ce..d6dd62d9b0cd 100644 --- a/release/Makefile +++ b/release/Makefile @@ -137,7 +137,9 @@ src.txz: mkdir -p ${DISTDIR}/usr ln -fs ${WORLDDIR} ${DISTDIR}/usr/src cd ${DISTDIR} && tar cLvf - --exclude .svn --exclude .zfs \ - --exclude .git --exclude @ --exclude usr/src/release/dist usr/src | \ + --exclude .git --exclude @ --exclude usr/src/release/dist \ + --exclude "usr/src/release/*.txz" \ + --exclude "usr/src/release/*.orig" usr/src | \ ${XZ_CMD} > ${.OBJDIR}/src.txz ports.txz: -- 2.37.1 From 05ff954893ceddefd4f6bfa4a74fbfcdea0c4ee2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 14 May 2015 12:07:25 +0200 Subject: [PATCH 051/310] Make reproducing builds more convenient ... by setting the various variables based on the environment variable REPRO_SEED. Obtained from: ElectroBSD --- Makefile.inc1 | 14 ++++++-- release/Makefile | 2 ++ share/mk/src.reproducible-build.mk | 54 ++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+), 3 deletions(-) create mode 100644 share/mk/src.reproducible-build.mk diff --git a/Makefile.inc1 b/Makefile.inc1 index b080f8a0b464..1af674adad76 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -56,6 +56,8 @@ _MKSHOWCONFIG= t .endif +.include "share/mk/src.reproducible-build.mk" + SRCDIR?= ${.CURDIR} LOCALBASE?= /usr/local @@ -1181,7 +1183,7 @@ buildworld: buildworld_prologue ${WMAKE_TGTS} buildworld_epilogue .PHONY _ncpu_cmd=sysctl -n hw.ncpu 2>/dev/null || nproc 2>/dev/null || echo unknown -buildworld_prologue: .PHONY +buildworld_prologue: .PHONY reproducible_build_hint @echo "--------------------------------------------------------------" @echo ">>> World build started on `LC_ALL=C date`" .if ${TARGET:Mmips} @@ -1190,7 +1192,7 @@ buildworld_prologue: .PHONY .endif @echo "--------------------------------------------------------------" -buildworld_epilogue: .PHONY +buildworld_epilogue: .PHONY reproducible_build_hint @echo @echo "--------------------------------------------------------------" @echo ">>> World build completed on `LC_ALL=C date`" @@ -1203,6 +1205,12 @@ buildworld_epilogue: .PHONY .endif @echo "--------------------------------------------------------------" +reproducible_build_hint: + @echo "--------------------------------------------------------------" + @echo ">>> To reproduce this build:" + @echo ">>> export REPRO_SEED=$${REPRO_SEED}" + @echo "--------------------------------------------------------------" + # # We need to have this as a target because the indirection between Makefile # and Makefile.inc1 causes the correct PATH to be used, rather than a @@ -1677,7 +1685,7 @@ _BUILDKERNEL_START!= date '+%s' # # Builds all kernels defined by BUILDKERNELS. # -buildkernel: .MAKE .PHONY +buildkernel: .MAKE .PHONY reproducible_build_hint .if empty(BUILDKERNELS:Ndummy) @echo "ERROR: Missing kernel configuration file(s) (${KERNCONF})."; \ false diff --git a/release/Makefile b/release/Makefile index d6dd62d9b0cd..3f0186f1ae7f 100644 --- a/release/Makefile +++ b/release/Makefile @@ -34,6 +34,8 @@ # TARGET/TARGET_ARCH: architecture of built release # +.include "../share/mk/src.reproducible-build.mk" + WORLDDIR?= ${.CURDIR}/.. PORTSDIR?= /usr/ports RELNOTES_LANG?= en_US.ISO8859-1 diff --git a/share/mk/src.reproducible-build.mk b/share/mk/src.reproducible-build.mk new file mode 100644 index 000000000000..dc4f4df93507 --- /dev/null +++ b/share/mk/src.reproducible-build.mk @@ -0,0 +1,54 @@ +########################################################################## +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## +# +# Make the build reproducible by exporting a bunch of variables, +# potentionally using an already-set REPRO_SEED as input. +# +# The variable names are somewhat stupid, mostly because we +# are using existing ones. +# +########################################################################## + +TZ= "UTC" +.export TZ + +.if ! defined(REPRO_SEED) +# XXX: Currently we don't add the kernel version number to the repro +# seed because we expect a clean object tree in which case it +# will reproducible be 0. If the object tree of a the +# build-to-reproduce was actually unclean, KERNEL_VERSION_NUMBER +# has to be set to a matching value. +REPRO_SEED!= echo $$(id -un):$$(hostname):$$(date +%s) +.export REPRO_SEED +.else +.if ! defined(KERNEL_VERSION_NUMBER) +KERNEL_VERSION_NUMBER=0 +.export KERNEL_VERSION_NUMBER +.endif +.endif + +USER!= echo "${REPRO_SEED}" | /usr/bin/cut -d : -f 1 +HOSTNAME!= echo "${REPRO_SEED}" | /usr/bin/cut -d : -f 2 +EPOCH_DATE!= echo "${REPRO_SEED}" | /usr/bin/cut -d : -f 3 + +DATE!= date -r ${EPOCH_DATE} +# These two probably are no longer necessary after r285701 +MKREPRO_DATE!= date -r ${EPOCH_DATE} +"%b %d %Y" +MKREPRO_TIME!= date -r ${EPOCH_DATE} +%H:%M:%S + +.for v in REPRO_SEED USER HOSTNAME EPOCH_DATE DATE MKREPRO_DATE MKREPRO_TIME +.export $v +.endfor -- 2.37.1 From b7358bf3b385c0e6ef88742598e19e4f7918d30e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 24 May 2015 18:01:31 +0200 Subject: [PATCH 052/310] Fake modification time smarter for everything but EXTRA_PACKAGES (XXX) While at it, remove duplicated slashes in the METALOG as they result in missing files. XXX 2017-01: It's not clear if this is still the case, but embedding duplicated slashes in the dist files is also aesthetically unpleasing. Mark two suspicious mtree spec modifications as such. XXX: investigate. Obtained from: ElectroBSD --- Makefile.inc1 | 20 ++++++++++++++++++++ release/Makefile | 8 ++++++-- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/Makefile.inc1 b/Makefile.inc1 index 1af674adad76..c4a7b38b4cbe 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1462,6 +1462,14 @@ distributeworld installworld stageworld: _installcheck_world .PHONY find ${DESTDIR}/${DISTDIR}/${dist} -mindepth 1 -type d -empty -delete .endfor .if defined(NO_ROOT) + @# Post process METALOG: add fake timestamps and, if necessary, + @# remove duplicated slashes which can occur if DISTDIR is undefined. + @# Keeping them results in missing files in the distribution tarballs. +.if defined(EPOCH_DATE) + sed -E -e 's@time=[0-9\.]+@@' \ + -e 's@(type=)@uid=0 gid=0 time=${EPOCH_DATE}.0 \1@' \ + -e 's@//@/@g' -i '.bak' ${METALOG} +.endif .for dist in base ${EXTRA_DISTRIBUTIONS} @# For each file that exists in this dist, print the corresponding @# line from the METALOG. This relies on the fact that @@ -1832,6 +1840,12 @@ distributekernel distributekernel.debug: .PHONY packagekernel: .PHONY .if defined(NO_ROOT) .if !defined(NO_INSTALLKERNEL) +.if defined(EPOCH_DATE) +# XXX: Is this really necessary given that we already modify the METALOG itself? + sed -E -e 's@time=[0-9\.]+@@' -e 's@(type=)@time=${EPOCH_DATE}.0 \1@' \ + -e 's@//@/@g' \ + -i '.bak' ${DESTDIR}/${DISTDIR}/kernel.meta +.endif cd ${DESTDIR}/${DISTDIR}/kernel; \ tar cvf - --exclude '*.debug' \ @${DESTDIR}/${DISTDIR}/kernel.meta | \ @@ -1845,6 +1859,12 @@ packagekernel: .PHONY .endif .if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes" .for _kernel in ${BUILDKERNELS:[2..-1]} +# XXX: See XXX above +.if defined(EPOCH_DATE) + sed -E -e 's@time=[0-9\.]+@@' -e 's@(type=)@time=${EPOCH_DATE}.0 \1@' \ + -e 's@//@/@g' \ + -i '.bak' ${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta +.endif cd ${DESTDIR}/${DISTDIR}/kernel.${_kernel}; \ tar cvf - --exclude '*.debug' \ @${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta | \ diff --git a/release/Makefile b/release/Makefile index 3f0186f1ae7f..eb0f287a0d8c 100644 --- a/release/Makefile +++ b/release/Makefile @@ -53,7 +53,11 @@ TARGET_ARCH= ${TARGET} IMAKE= ${MAKE} TARGET_ARCH=${TARGET_ARCH} TARGET=${TARGET} DISTDIR= dist -# Define OSRELEASE by using newvers.sh +# Enable mtree spec usage so we can fake the modification time. +NO_ROOT=1 +.export NO_ROOT + +# Define OSRELEASE by using newvars.sh .if !defined(OSRELEASE) || empty(OSRELEASE) .for _V in TYPE BRANCH REVISION ${_V}!= eval $$(awk '/^${_V}=/{print}' ${.CURDIR}/../sys/conf/newvers.sh); echo $$${_V} @@ -242,7 +246,7 @@ mini-memstick.img: bootonly sh ${.CURDIR}/${TARGET}/make-memstick.sh bootonly ${.TARGET} packagesystem: base.txz kernel.txz ${EXTRA_PACKAGES} - for tarball in *.txz; do \ + for tarball in ${EXTRA_PACKAGES}; do \ sh ${.CURDIR}/scripts/tar-time-reset.sh $${tarball}; \ done sh ${.CURDIR}/scripts/make-manifest.sh *.txz > MANIFEST -- 2.37.1 From 12bfcc22e49e4fa57d7113f175f50042a575814f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Jun 2015 13:56:05 +0200 Subject: [PATCH 053/310] release/Makefile: Build the src.txz only once ... and also enforce reproducible source ownership and permissions. If taring the sources fails, keep the mtree spec. Keep release/scripts/tar-time-reset.sh for now but update a comment to make it obvious that the script isn't used anymore. NB: A previos version of this commit gave directories in the source tarball 550 permissions. This seems reasonable at first glance but breaks poudriere builds with unprivileges users. Poudriere applies the permissions from the source tarball to /usr, thus making it unaccesible for users other than root and members of wheel. While only giving /usr 555 permissions is sufficient to prevent the issue, treating the directory is more work and as the source is public anyway it didn't seem worth it. Obtained from: ElectroBSD --- release/Makefile | 31 +++++++++++++++++++++++-------- release/scripts/tar-time-reset.sh | 8 +++----- 2 files changed, 26 insertions(+), 13 deletions(-) diff --git a/release/Makefile b/release/Makefile index eb0f287a0d8c..76a4ca0632e7 100644 --- a/release/Makefile +++ b/release/Makefile @@ -142,11 +142,29 @@ kernel.txz: src.txz: mkdir -p ${DISTDIR}/usr ln -fs ${WORLDDIR} ${DISTDIR}/usr/src - cd ${DISTDIR} && tar cLvf - --exclude .svn --exclude .zfs \ - --exclude .git --exclude @ --exclude usr/src/release/dist \ - --exclude "usr/src/release/*.txz" \ - --exclude "usr/src/release/*.orig" usr/src | \ - ${XZ_CMD} > ${.OBJDIR}/src.txz +# Create an mtree spec with faked timestamps so we get a reproducible +# tar file. We do not use tar for this because its mtree generator +# appears to be buggy and exits with an memory allocation failure. +# +# It's important that the excluded paths start with "./", otherwise +# file locations are not recorded correctly, and, for example, +# usr/src/usr.sbin appears as usr/src/release/usr.sbin in the +# tar file. Only the shadow knows if that's a bug or a feature. + echo "./usr/src/.git" >${.OBJDIR}/mtree-exclude + echo "./usr/src/release/dist" >>${.OBJDIR}/mtree-exclude + echo "./usr/src/release/src.mtree" >>${.OBJDIR}/mtree-exclude + echo "./usr/src/release/mtree-exclude" >>${.OBJDIR}/mtree-exclude + cd ${DISTDIR} && mtree -c -L -k time -X ${.OBJDIR}/mtree-exclude | \ + mtree -C | \ + sed -E -e 's@time=[0-9]+\.[0-9]+@time=${EPOCH_DATE}.0 uid=0 gid=0@' \ + -e 's@(type=dir)@\1 mode=0555@' \ + -e 's@(type=file)@\1 mode=0444@' \ + -e 's@//@/@g' \ + > ${.OBJDIR}/src.mtree + rm ${.OBJDIR}/mtree-exclude + cd ${DISTDIR} && tar cLvf - @${.OBJDIR}/src.mtree \ + | ${XZ_CMD} > ${.OBJDIR}/src.txz && \ + rm ${.OBJDIR}/src.mtree ports.txz: mkdir -p ${DISTDIR}/usr @@ -246,9 +264,6 @@ mini-memstick.img: bootonly sh ${.CURDIR}/${TARGET}/make-memstick.sh bootonly ${.TARGET} packagesystem: base.txz kernel.txz ${EXTRA_PACKAGES} - for tarball in ${EXTRA_PACKAGES}; do \ - sh ${.CURDIR}/scripts/tar-time-reset.sh $${tarball}; \ - done sh ${.CURDIR}/scripts/make-manifest.sh *.txz > MANIFEST touch ${.TARGET} diff --git a/release/scripts/tar-time-reset.sh b/release/scripts/tar-time-reset.sh index 8186fd8d3e26..f1b84340286d 100755 --- a/release/scripts/tar-time-reset.sh +++ b/release/scripts/tar-time-reset.sh @@ -17,11 +17,9 @@ ########################################################################## # # This script resets the timestamps in a given tarfile to hopefully make -# it reproducible. This is a rather wasteful approach, but works for now. -# -# A better solution would be to patch bsdtar to optionally use a fixed -# time (without having to clown around with mtree specs). -# +# it reproducible. As this is a rather wasteful approach the script is +# no longer used. It hasn't been removed yet as it may be useful for +# testing purposes. ########################################################################## main() { -- 2.37.1 From f5069b944691aeccacd61b31c6cdcf4496fa0cb2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 25 May 2015 10:27:48 +0200 Subject: [PATCH 054/310] Add image-checksum.sh ... which calculates a checksum of the reproducible parts of an memstick image. Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 156 ++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100755 release/scripts/image-checksum.sh diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh new file mode 100755 index 000000000000..6f79a81ae37a --- /dev/null +++ b/release/scripts/image-checksum.sh @@ -0,0 +1,156 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## +# +# image-checksum.sh /path/to/memstick.img +# +# Unfortunately the memstick target currently does not create reproducible +# ElectroBSD images due to unreproducible differences in the file system +# layer of the data partition. +# +# To be able to (sort of) compare memstick images anyway, this script +# produces a "partial image checksum" that is based on the partition layout, +# the checksum of the boot code partition and an mtree spec of the data +# partition which includes checksums, sizes and timestamps for all the +# files. +# +# A memstick image whose "partial checksum" matches the one of another +# image can be totally considered to be nearly as trustworthy. Obviously +# that's a somewhat worthless property, it is thus recommended that you +# rebuild the potentionally malicious image using a trusted operating +# system first. After you've done this, potentionally malicious differences +# in the unchecked parts should be gone. +# +# Just kidding, image-checksum.sh is only intended to regression-test +# the ElectroBSD build system. +# +# Also note that this script relies on non-standardized output of other +# tools which might occasionally change. To be able to reproduce partial +# image checksums you thus need a userland that is close enough to the +# one that was used to create the original version. +# +########################################################################## + +UFS_PARTITION=p2 +EXPECTED_PARTITIONS=2 +MOUNTPOINT=/mnt +VERBOSE=0 +MTREE_KEYWORDS=size,time,uid,gid,sha256 + +verbose_log() { + local message="$*" + if [ "${VERBOSE}" = 0 ]; then + return + fi + echo "${message}" +} + +create_mtree_spec_file() { + local md_unit spec_file + + md_unit="${1}" + spec_file="${2}" + + verbose_log "Mounting /dev/md${md_unit}${UFS_PARTITION} at ${MOUNTPOINT}" + mount -o ro "/dev/md${md_unit}${UFS_PARTITION}" "${MOUNTPOINT}" || return 1 + + verbose_log "Running mtree, saving spec in ${spec_file}" + mtree -c -k "${MTREE_KEYWORDS}" -p "${MOUNTPOINT}" | mtree -C -k all > "${spec_file}" || return 1 + + verbose_log "Unmounting ${MOUNTPOINT} ..." + umount "${MOUNTPOINT}" || return 1 +} + +partition_count_acceptable() { + local md_unit="${1}" + + # Verify that there are exactly two partitions present + partitions=$(gpart show -r -p "md${md_unit}" | grep -c "md${md_unit}"p) + if [ "${partitions}" != "${EXPECTED_PARTITIONS}" ]; then + echo "Invalid number of partitions: ${partitions}" + return 1; + fi +} + +main() { + local image_file \ + args md_unit spec_file gpart_file + + args=$(getopt v $*) + if [ $? -ne 0 ]; then + echo 'You are doing it wrong: Invalid flag specified' + exit 2 + fi + set -- ${args} + while true; do + case "$1" in + -v) + VERBOSE=1 + shift + ;; + --) + shift; break + ;; + esac + done + + image_file=${1} + if [ -z "${image_file}" ]; then + echo "No image file provided" + return 1 + fi + spec_file="${image_file}.mtree" + if [ -f "${spec_file}" ]; then + echo "Spec file ${spec_file} already exists" + return 1 + fi + gpart_file="${image_file}.gpart" + if [ -f "${spec_file}" ]; then + echo "gpart file ${gpart_file} already exists" + return 1 + fi + + md_unit=$(mdconfig -o readonly -n -f "${image_file}") + if [ $? != 0 ]; then + return 1 + fi + + partition_count_acceptable "${md_unit}" || return 1 + + if [ ! -f "${spec_file}" ]; then + create_mtree_spec_file "${md_unit}" "${spec_file}" || return 1 + fi + if [ ! -f "${gpart_file}" ]; then + gpart list "md${md_unit}" | sed -E -e "s@(: md)${md_unit}@\1X@" > "${gpart_file}" + fi + + gpart_checksum=$(sha256 -q "${gpart_file}") + verbose_log "gpart checksum: ${gpart_checksum}" + + mdconfig -d -u "${md_unit}" || return 1 + + bootcode_checksum=$(dd if=/dev/md${md_unit}p1 2>/dev/null | sha256) + verbose_log "Boot code checksum: ${bootcode_checksum}" + mtree_checksum=$(sha256 -q "${spec_file}") + verbose_log "mtree checksum: ${mtree_checksum}" + + weak_image_checksum=$(echo "${gpart_checksum} ${bootcode_checksum} ${mtree_checksum}" | sha256) + echo "Partial image checksum: ${weak_image_checksum}" + +} + +main "${@}" -- 2.37.1 From c37f04cedccb87f21827993e6bf3105e1b9be849 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 3 Jul 2015 19:54:10 +0200 Subject: [PATCH 055/310] image-checksum: Add -r flag to reuse cache files Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 6f79a81ae37a..1940921b44a1 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -50,6 +50,7 @@ EXPECTED_PARTITIONS=2 MOUNTPOINT=/mnt VERBOSE=0 MTREE_KEYWORDS=size,time,uid,gid,sha256 +REUSE_EXISTING_CACHE_FILES=false verbose_log() { local message="$*" @@ -90,7 +91,7 @@ main() { local image_file \ args md_unit spec_file gpart_file - args=$(getopt v $*) + args=$(getopt rv $*) if [ $? -ne 0 ]; then echo 'You are doing it wrong: Invalid flag specified' exit 2 @@ -98,6 +99,10 @@ main() { set -- ${args} while true; do case "$1" in + -r) + REUSE_EXISTING_CACHE_FILES=true + shift + ;; -v) VERBOSE=1 shift @@ -116,12 +121,12 @@ main() { spec_file="${image_file}.mtree" if [ -f "${spec_file}" ]; then echo "Spec file ${spec_file} already exists" - return 1 + ${REUSE_EXISTING_CACHE_FILES} || return 1 fi gpart_file="${image_file}.gpart" if [ -f "${spec_file}" ]; then echo "gpart file ${gpart_file} already exists" - return 1 + ${REUSE_EXISTING_CACHE_FILES} || return 1 fi md_unit=$(mdconfig -o readonly -n -f "${image_file}") -- 2.37.1 From 361d33f1464d734c96c35187c2c80dedf092f30f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 3 Jul 2015 20:20:01 +0200 Subject: [PATCH 056/310] image-checksum.sh: Allow to overwrite the mtree flags Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 1940921b44a1..cc31e9055fe3 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -91,7 +91,7 @@ main() { local image_file \ args md_unit spec_file gpart_file - args=$(getopt rv $*) + args=$(getopt m:rv $*) if [ $? -ne 0 ]; then echo 'You are doing it wrong: Invalid flag specified' exit 2 @@ -99,6 +99,11 @@ main() { set -- ${args} while true; do case "$1" in + -m) + shift + MTREE_KEYWORDS="${1}" + shift + ;; -r) REUSE_EXISTING_CACHE_FILES=true shift -- 2.37.1 From 7556e75214e74cc48d40a3ae7e4d122da2ed533f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 31 Aug 2015 10:51:16 +0200 Subject: [PATCH 057/310] release/scripts/image-checksum.sh: Allow to checksum multiple images at once Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 79 +++++++++++++++++-------------- 1 file changed, 44 insertions(+), 35 deletions(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index cc31e9055fe3..30ac62a6c168 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -87,42 +87,11 @@ partition_count_acceptable() { fi } -main() { +generate_partial_image_checksum() { local image_file \ - args md_unit spec_file gpart_file - - args=$(getopt m:rv $*) - if [ $? -ne 0 ]; then - echo 'You are doing it wrong: Invalid flag specified' - exit 2 - fi - set -- ${args} - while true; do - case "$1" in - -m) - shift - MTREE_KEYWORDS="${1}" - shift - ;; - -r) - REUSE_EXISTING_CACHE_FILES=true - shift - ;; - -v) - VERBOSE=1 - shift - ;; - --) - shift; break - ;; - esac - done + md_unit spec_file gpart_file - image_file=${1} - if [ -z "${image_file}" ]; then - echo "No image file provided" - return 1 - fi + image_file="${1}" spec_file="${image_file}.mtree" if [ -f "${spec_file}" ]; then echo "Spec file ${spec_file} already exists" @@ -159,8 +128,48 @@ main() { verbose_log "mtree checksum: ${mtree_checksum}" weak_image_checksum=$(echo "${gpart_checksum} ${bootcode_checksum} ${mtree_checksum}" | sha256) - echo "Partial image checksum: ${weak_image_checksum}" + echo "Partial image checksum for ${image_file}: ${weak_image_checksum}" +} +main() { + local image_file \ + args + + args=$(getopt m:rv $*) + if [ $? -ne 0 ]; then + echo 'You are doing it wrong: Invalid flag specified' + exit 2 + fi + set -- ${args} + while true; do + case "$1" in + -m) + shift + MTREE_KEYWORDS="${1}" + shift + ;; + -r) + REUSE_EXISTING_CACHE_FILES=true + shift + ;; + -v) + VERBOSE=1 + shift + ;; + --) + shift; break + ;; + esac + done + + if [ -z "${1}" ]; then + echo "No image file provided" + return 1 + fi + + for image_file in "${@}"; do + generate_partial_image_checksum "${image_file}" || return 1 + done } main "${@}" -- 2.37.1 From 25cecc191e94be7a40676be2c5b353b0549c2721 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 30 May 2015 14:38:48 +0200 Subject: [PATCH 058/310] sys: Do not embed the compiler version in the kernel binary ... as it makes reproducing the binary with a different compiler more complicated. In case of ElectroBSD the compiler used can be usually deduced from the uname output anyway as the upstream revision is part of the fake hostname when using reproduce.sh. Obtained from: ElectroBSD --- sys/conf/newvers.sh | 2 -- sys/kern/init_main.c | 1 - sys/kern/kern_mib.c | 3 --- sys/sys/systm.h | 1 - 4 files changed, 7 deletions(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index 1c5ecf28f97a..dc55af24a097 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -194,7 +194,6 @@ else t=$(date) fi i=$(${MAKE:-make} -V KERN_IDENT) -compiler_v=$($(${MAKE:-make} -V CC) -v 2>&1 | grep -w 'version') for dir in /usr/bin /usr/local/bin; do if [ ! -z "${svnversion}" ] ; then @@ -311,7 +310,6 @@ $COPYRIGHT char sccs[sizeof(SCCSSTR) > 128 ? sizeof(SCCSSTR) : 128] = SCCSSTR; char version[sizeof(VERSTR) > 256 ? sizeof(VERSTR) : 256] = VERSTR; -char compiler_version[] = "${compiler_v}"; char ostype[] = "${TYPE}"; char osrelease[sizeof(RELSTR) > 32 ? sizeof(RELSTR) : 32] = RELSTR; int osreldate = ${RELDATE}; diff --git a/sys/kern/init_main.c b/sys/kern/init_main.c index c8cb0f5613ee..55a9e6980c9e 100644 --- a/sys/kern/init_main.c +++ b/sys/kern/init_main.c @@ -361,7 +361,6 @@ print_version(void *data __unused) while (len > 0 && version[len - 1] == '\n') len--; printf("%.*s %s\n", len, version, machine); - printf("%s\n", compiler_version); } SYSINIT(announce, SI_SUB_COPYRIGHT, SI_ORDER_FIRST, print_caddr_t, diff --git a/sys/kern/kern_mib.c b/sys/kern/kern_mib.c index 483bbe453b0c..5db99784311c 100644 --- a/sys/kern/kern_mib.c +++ b/sys/kern/kern_mib.c @@ -105,9 +105,6 @@ SYSCTL_INT(_kern, KERN_OSREV, osrevision, CTLFLAG_RD|CTLFLAG_CAPRD, SYSCTL_STRING(_kern, KERN_VERSION, version, CTLFLAG_RD|CTLFLAG_MPSAFE, version, 0, "Kernel version"); -SYSCTL_STRING(_kern, OID_AUTO, compiler_version, CTLFLAG_RD|CTLFLAG_MPSAFE, - compiler_version, 0, "Version of compiler used to compile kernel"); - SYSCTL_STRING(_kern, KERN_OSTYPE, ostype, CTLFLAG_RD|CTLFLAG_MPSAFE| CTLFLAG_CAPRD, ostype, 0, "Operating system type"); diff --git a/sys/sys/systm.h b/sys/sys/systm.h index b2b84d0fc722..dc722d13d462 100644 --- a/sys/sys/systm.h +++ b/sys/sys/systm.h @@ -55,7 +55,6 @@ extern int cold; /* nonzero if we are doing a cold boot */ extern int suspend_blocked; /* block suspend due to pending shutdown */ extern int rebooting; /* kern_reboot() has been called. */ extern char version[]; /* system version */ -extern char compiler_version[]; /* compiler version */ extern char copyright[]; /* system copyright */ extern int kstack_pages; /* number of kernel stack pages */ -- 2.37.1 From f1d224809729e1f8366433b7c060485e429453e5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Jun 2015 12:18:16 +0200 Subject: [PATCH 059/310] release/Makefile: Reorder dependencies to reduce the chances that base or kernel parts end up in the src.txz. This is merely a workaround Obtained from: ElectroBSD --- release/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/Makefile b/release/Makefile index 76a4ca0632e7..6ef295328738 100644 --- a/release/Makefile +++ b/release/Makefile @@ -263,7 +263,7 @@ mini-memstick: mini-memstick.img mini-memstick.img: bootonly sh ${.CURDIR}/${TARGET}/make-memstick.sh bootonly ${.TARGET} -packagesystem: base.txz kernel.txz ${EXTRA_PACKAGES} +packagesystem: ${EXTRA_PACKAGES} base.txz kernel.txz sh ${.CURDIR}/scripts/make-manifest.sh *.txz > MANIFEST touch ${.TARGET} -- 2.37.1 From 751dcf8da8d1b39409fc4362ee9b4ecd1004c3f0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 17:30:25 +0200 Subject: [PATCH 060/310] Add strip-freebsd.sh ... which suggests a bunch of stuff to delete from a vanilla FreeBSD checkout. In a previous life it was called free-freebsd.sh which obviously sounds more awesome, but nowadays most of the suggested stuff for removal is actually free software that just isn't relevant for ElectroBSD. Removing code we don't need means we don't have to care about its security and license issues. There's lots of code to remove left! Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 183 +++++++++++++++++++++++++++++++ 1 file changed, 183 insertions(+) create mode 100755 release/scripts/strip-freebsd.sh diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh new file mode 100755 index 000000000000..e805f9cab8e4 --- /dev/null +++ b/release/scripts/strip-freebsd.sh @@ -0,0 +1,183 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## + +# This script pretends to free a FreeBSD checkout from +# known-unfree files and other stuff that is not required +# by ElectroBSD. +# +# While it is pretty much guaranteed to ditch a bunch of files +# it doesn't work very thorougly and the resulting checkout is +# likely to still contain lots of non-free parts that haven't +# been discovered yet. + +get_snd_csa_files() { + find sys/dev/sound/pci -name "csa*" +} + +get_snd_ds1_files() { + find sys/dev/sound/pci -name "ds1*" +} + +get_snd_maestro3_files() { + # This file contains the actual blobs + echo sys/dev/sound/pci/allegro_code.h + + # This file contains the (free) code that relies on the blobs. + echo sys/dev/sound/pci/maestro3.c +} + +# We can't simply remove the whole directory +# as a (free) header is required by bge. +get_bce_files() { + echo "sys/dev/bce/if_bcefw.h" \ + "sys/dev/bce/if_bce.c" +} + +get_usb_firmware_files() { + find sys/dev/usb/ -name "*fw*" +} + +# XXX: Misleading name, some of the files are merely tainted +# by non-free dependencies +get_unfree_files() { + find sys/ -name "*.uu" + get_snd_csa_files + get_snd_ds1_files + get_snd_maestro3_files + get_bce_files + get_usb_firmware_files +} + +get_files_to_ditch() { + get_unfree_files +} + +# These architectures are mainly unsupported by ElectroBSD +# due to lack of hardware for testing purposes. +# +# The source directories are mainly removed to shrink the +# source tarball and to reduce the number of files that +# should be audited for license and security issues. +get_unsupported_architectures() { + echo "arm arm64 mips pc98 powerpc riscv sparc64" +} + +# These depend on or contain proprietary firmware that is included in sys/contrib/dev +get_tainted_sys_contrib_devs() { + echo "drm2 ipw iwi iwm iwn mwl npe otus ral rsu rtwn run uath urtwn wpi" +} + +# These require proprietary firmware that is included in sys/dev +# and may cause build failures without it. +get_tainted_sys_devs() { + # bce has already been taken care of by get_bce_files() above + echo "bxe ctau cx cxgb cxgbe ispfw qlxgbe" \ + "it tw" +} + +get_unused_contrib_dirs() { + # XXX: gcc can't be deleted because parts of it are apparently + # required to build libc. This should be investigated more thoroughly, + # hopefully it can be fixed. + echo "apr apr-util ipfilter ofed sendmail serf subversion tcsh" +} + +# Only includes directory that aren't architecture-specific +get_unused_cross_platform_sys_dirs() { + echo "netnatm ofed" +} + +get_directories_to_ditch() { + local arch \ + dir arch_dir sys_contrib contrib_dir + + for dir in sys sys/boot; do + for arch in $(get_unsupported_architectures); do + potential_directory="${dir}/${arch}" + if [ -d "${potential_directory}" ]; then + echo "${potential_directory}" + fi + done + done + + for dir in $(get_unused_cross_platform_sys_dirs); do + potential_directory="sys/${dir}" + if [ -d "${potential_directory}" ]; then + echo "${potential_directory}" + fi + done + + for sys_contrib in ipfilter ncsw octeon-sdk; do + echo "sys/contrib/${sys_contrib}" + done + + for sys_contrib in $(get_tainted_sys_contrib_devs); do + echo "sys/contrib/dev/${sys_contrib}" + done + + for sys_dev in $(get_tainted_sys_devs); do + echo "sys/dev/${sys_dev}" + done + + for contrib_dir in $(get_unused_contrib_dirs); do + potential_directory="contrib/${contrib_dir}" + if [ -d "${potential_directory}" ]; then + echo "${potential_directory}" + fi + done +} + +purify_cwd() { + # There are no spaces in paths or file names. + files_to_ditch="$(get_files_to_ditch)" + for f in $files_to_ditch; do + [ -f "${f}" ] && echo "rm ${f}" + done + + dirs_to_ditch="$(get_directories_to_ditch)" + for d in $dirs_to_ditch; do + [ -d "${d}" ] && echo "rm -r ${d}" + done +} + +main() { + local src_dir \ + files_to_ditch dirs_to_ditch + + src_dir="${1}" + if [ -z "${src_dir}" ]; then + echo "No source directory given" + return 1 + fi + if [ ! -d "${src_dir}" ]; then + echo "No such directory: ${src_dir}" + return 1 + fi + # Make it less likely to operate on a directory + # that isn't actually a FreeBSD checkout + if [ ! -f "${src_dir}/COPYRIGHT" ]; then + echo "${src_dir} contains no COPYRIGHT file" + return 1 + fi + + cd "${src_dir}" || return 1 + + purify_cwd +} + +main "${@}" -- 2.37.1 From 237392cf4f8b5b3438c1eafd3a6d8f8520359ceb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 7 Dec 2016 12:25:38 +0100 Subject: [PATCH 061/310] Detach telnetd from the build Obtained from: ElectroBSD --- secure/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/secure/Makefile b/secure/Makefile index 198df326fe32..29d7f7b13ef8 100644 --- a/secure/Makefile +++ b/secure/Makefile @@ -12,7 +12,7 @@ SUBDIR.${MK_CAROOT}+= caroot # These are the programs which depend on crypto, but not Kerberos. SPROGS= lib/libfetch lib/libpam lib/libradius lib/libtelnet \ - bin/ed libexec/telnetd usr.bin/fetch usr.bin/telnet \ + bin/ed usr.bin/fetch usr.bin/telnet \ usr.sbin/ppp usr.sbin/tcpdump/tcpdump .if ${MK_SENDMAIL} != "no" SPROGS+=usr.sbin/sendmail -- 2.37.1 From ac20047157fe83cc4ebf19eea03e2de90b59585c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 4 Nov 2015 18:55:18 +0100 Subject: [PATCH 062/310] share/doc: Detach 'legal' from the build It (tries to) install proprietary licenses for code that is not part of ElectroBSD and deleted by reproduce.sh's auto-untaint mode (-a). Obtained from: ElectroBSD --- share/doc/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/share/doc/Makefile b/share/doc/Makefile index 7718c6344dc4..80bdc6843034 100644 --- a/share/doc/Makefile +++ b/share/doc/Makefile @@ -5,7 +5,6 @@ SUBDIR= ${_IPv6} \ ${_atf} \ - legal \ ${_llvm} \ ${_pjdfstest} \ ${_roffdocs} -- 2.37.1 From a0b8365d7bf95cd9f4fc55b65dfa85bef86d4e1e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 21 May 2015 18:42:42 +0200 Subject: [PATCH 063/310] release/Makefile: Export VOLUME_LABEL for make-memstick.sh Obtained from: ElectroBSD --- release/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/release/Makefile b/release/Makefile index 6ef295328738..d12e799cbabc 100644 --- a/release/Makefile +++ b/release/Makefile @@ -77,6 +77,8 @@ VOLUME_LABEL= ${REVISION:C/[.-]/_/g}_${BRANCH:C/[.-]/_/g}_${TARGET_ARCH} VOLUME_LABEL= FreeBSD_Install .endif +.export VOLUME_LABEL + .if !exists(${PORTSDIR}) || !defined(WITH_PORTS) NOPORTS= true .endif -- 2.37.1 From 8b134d8c6a014a04cc1d0e55d4e4411b776ae0d8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 3 Dec 2015 18:34:21 +0100 Subject: [PATCH 064/310] lib/libkvm: Unbreak the build without kernel sources that don't ship with ElectroBSD Obtained from: ElectroBSD --- lib/libkvm/Makefile | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/lib/libkvm/Makefile b/lib/libkvm/Makefile index 7747518d05eb..1ffda664faaa 100644 --- a/lib/libkvm/Makefile +++ b/lib/libkvm/Makefile @@ -10,14 +10,8 @@ CFLAGS+=-DNO__SCCSID -I${.CURDIR} SRCS= kvm.c kvm_cptime.c kvm_getloadavg.c \ kvm_getswapinfo.c kvm_pcpu.c kvm_private.c kvm_proc.c kvm_vnet.c \ - kvm_minidump_aarch64.c \ kvm_amd64.c kvm_minidump_amd64.c \ - kvm_arm.c kvm_minidump_arm.c \ - kvm_i386.c kvm_minidump_i386.c \ - kvm_minidump_mips.c \ - kvm_powerpc.c kvm_powerpc64.c \ - kvm_minidump_riscv.c \ - kvm_minidump_powerpc64.c kvm_minidump_powerpc64_hpt.c + kvm_i386.c kvm_minidump_i386.c INCS= kvm.h LIBADD= elf -- 2.37.1 From 6385981b6f84cab0dc5bb1f1ac5690fea77ddddc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 24 May 2015 14:03:06 +0200 Subject: [PATCH 065/310] usr.sbin/pkg: Change URL_SCHEME_PREFIX to an URL more easily controlled by the user ... through ssh port forwarding. Obtained from: ElectroBSD --- usr.sbin/pkg/config.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/usr.sbin/pkg/config.c b/usr.sbin/pkg/config.c index 08e206b93511..b9f5356fc560 100644 --- a/usr.sbin/pkg/config.c +++ b/usr.sbin/pkg/config.c @@ -66,7 +66,7 @@ static struct config_entry c[] = { [PACKAGESITE] = { PKG_CONFIG_STRING, "PACKAGESITE", - URL_SCHEME_PREFIX "http://pkg.FreeBSD.org/${ABI}/latest", + URL_SCHEME_PREFIX "http://127.0.0.1:8000/packages/${ABI}/", NULL, NULL, false, -- 2.37.1 From 0574e4e9bb46e2f397bba4f06ef6f103dfd60f86 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 25 May 2015 15:33:59 +0200 Subject: [PATCH 066/310] share/doc: Exclude directories that contain (partly) non-free materials Obtained from: ElectroBSD --- share/doc/Makefile | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/share/doc/Makefile b/share/doc/Makefile index 80bdc6843034..dc2003f7ecfe 100644 --- a/share/doc/Makefile +++ b/share/doc/Makefile @@ -3,11 +3,9 @@ .include -SUBDIR= ${_IPv6} \ - ${_atf} \ +SUBDIR= ${_atf} \ ${_llvm} \ - ${_pjdfstest} \ - ${_roffdocs} + ${_pjdfstest} .if ${MK_TESTS} != "no" _atf= atf @@ -18,9 +16,11 @@ _pjdfstest= pjdfstest _llvm= llvm .endif -.if ${MK_INET6} != "no" -_IPv6= IPv6 -.endif +# Note: Documents in the sub directories IPv6, papers, psd, smm +# and usd are not installed on ElectroBSD due to license problems. +# +# Some of the excluded papers actually have free licenses and +# should eventually be installed. SUBDIR_PARALLEL= -- 2.37.1 From 76b68c647082135e0f2ef5a7d2b1f2cfd2a08ded Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 May 2015 10:10:43 +0200 Subject: [PATCH 067/310] geom: Do not build raid, raid3 and vinum classes Obtained from: ElectroBSD --- sys/modules/geom/Makefile | 3 --- 1 file changed, 3 deletions(-) diff --git a/sys/modules/geom/Makefile b/sys/modules/geom/Makefile index 51f3d2438eb1..c62bbe7bb6e8 100644 --- a/sys/modules/geom/Makefile +++ b/sys/modules/geom/Makefile @@ -17,12 +17,9 @@ SUBDIR= geom_bde \ geom_multipath \ geom_nop \ geom_part \ - geom_raid \ - geom_raid3 \ geom_shsec \ geom_stripe \ geom_uzip \ - geom_vinum \ geom_virstor \ geom_zero -- 2.37.1 From 967942640eb90930235d35bfc7c529ac44207b61 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 May 2015 12:17:36 +0200 Subject: [PATCH 068/310] share/mk/src.opts.mk: Disable a bunch of options by default (XXX: Remove BSD_GREP_FASTMATCH) .. either due to license issues or because they are not considered relevant for the majority of ElectroBSD users. src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 61 +++++++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 24 deletions(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 3fb104ad1c50..ccd38715c495 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -57,25 +57,22 @@ __DEFAULT_YES_OPTIONS = \ ACPI \ APM \ AT \ - ATM \ AUDIT \ AUTHPF \ AUTOFS \ BHYVE \ BLACKLIST \ - BLUETOOTH \ BOOT \ BOOTPARAMD \ BOOTPD \ BSD_CPIO \ - BSDINSTALL \ + BSD_GREP_FASTMATCH \ BSNMP \ BZIP2 \ CALENDAR \ CAPSICUM \ CAROOT \ CASPER \ - CCD \ CDDL \ CLANG \ CLANG_BOOTSTRAP \ @@ -98,12 +95,8 @@ __DEFAULT_YES_OPTIONS = \ FDT \ FILE \ FINGER \ - FLOPPY \ - FMTREE \ FORTH \ FP_LIBC \ - FREEBSD_UPDATE \ - FTP \ GAMES \ GH_BC \ GNU_DIFF \ @@ -116,9 +109,7 @@ __DEFAULT_YES_OPTIONS = \ INET \ INET6 \ INETD \ - IPFILTER \ IPFW \ - ISCSI \ JAIL \ KDUMP \ KVM \ @@ -144,8 +135,7 @@ __DEFAULT_YES_OPTIONS = \ MAILWRAPPER \ MAKE \ MALLOC_PRODUCTION \ - MLX5TOOL \ - NDIS \ + MANDOCDB \ NETCAT \ NETGRAPH \ NLS_CATALOGS \ @@ -157,33 +147,23 @@ __DEFAULT_YES_OPTIONS = \ PAM \ PF \ PKGBOOTSTRAP \ - PMC \ - PORTSNAP \ - PPP \ QUOTAS \ RADIUS_SUPPORT \ - RBOOTD \ REPRODUCIBLE_BUILD \ RESCUE \ - ROUTED \ - SENDMAIL \ + RCS \ + RESCUE \ SERVICESDB \ SETUID_LOGIN \ SHARED_TOOLCHAIN \ SHAREDOCS \ - SOURCELESS \ - SOURCELESS_HOST \ - SOURCELESS_UCODE \ STATS \ SYSCONS \ SYSTEM_COMPILER \ SYSTEM_LINKER \ TALK \ - TCP_WRAPPERS \ - TCSH \ TELNET \ TEXTPROC \ - TFTP \ UNBOUND \ USB \ UTMPX \ @@ -243,6 +223,39 @@ __DEFAULT_DEPENDENT_OPTIONS= \ __DEFAULT_DEPENDENT_OPTIONS+= ${var}_SUPPORT/${var} .endfor +# Disable a bunch of additional options that default to yes in FreeBSD +__DEFAULT_NO_OPTIONS += \ + ATM \ + BLUETOOTH \ + BSDINSTALL \ + CCD \ + CTM \ + FLOPPY \ + FMTREE \ + FREEBSD_UPDATE \ + FTP \ + GCOV \ + IPFILTER \ + ISCSI \ + NDIS \ + RBOOTD \ + PC_SYSINSTALL \ + PMC \ + PORTSNAP \ + PPP \ + RCMDS \ + ROUTED \ + SENDMAIL \ + SVNLITE \ + SOURCELESS \ + SOURCELESS_HOST \ + SOURCELESS_UCODE \ + SYSINSTALL \ + TCP_WRAPPERS \ + TCSH \ + TFTP \ + TIMED \ + # # Default behaviour of some options depends on the architecture. Unfortunately # this means that we have to test TARGET_ARCH (the buildworld case) as well -- 2.37.1 From 1c9eb6fc381d3074d5eb2008365462d1570bceea Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 24 Oct 2015 12:58:42 +0200 Subject: [PATCH 069/310] share/mk/src.opts.mk: Disable EXAMPLES ... so we don't have to care about license issues. Some of the examples are non-free, in some cases the license is unclear. XXX: Some of the examples are properly licensed and moderately useful, for example share/examples/bhyve/vmrun.sh. It can be executed from a checkout directly, though. src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index ccd38715c495..8b4823a48b0d 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -91,7 +91,6 @@ __DEFAULT_YES_OPTIONS = \ EE \ EFI \ ELFTOOLCHAIN_BOOTSTRAP \ - EXAMPLES \ FDT \ FILE \ FINGER \ @@ -230,6 +229,7 @@ __DEFAULT_NO_OPTIONS += \ BSDINSTALL \ CCD \ CTM \ + EXAMPLES \ FLOPPY \ FMTREE \ FREEBSD_UPDATE \ -- 2.37.1 From d6dc86fb1a089501678d4751ba89c6e49e27a2ef Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 24 Jun 2015 12:53:20 +0200 Subject: [PATCH 070/310] share/mk/src.opts.mk: Enable CLANG_EXTRAS by default as it's required for llvm-symbolizer (XXX: Doubled entry) src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 8b4823a48b0d..992a397735a1 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -174,10 +174,13 @@ __DEFAULT_YES_OPTIONS = \ LOADER_ZFS \ ZONEINFO +# Enable additional options that default to NO in FreeBSD +__DEFAULT_YES_OPTIONS += \ + CLANG_EXTRAS \ + __DEFAULT_NO_OPTIONS = \ BEARSSL \ BHYVE_SNAPSHOT \ - CLANG_EXTRAS \ CLANG_FORMAT \ DTRACE_TESTS \ EXPERIMENTAL \ @@ -222,6 +225,10 @@ __DEFAULT_DEPENDENT_OPTIONS= \ __DEFAULT_DEPENDENT_OPTIONS+= ${var}_SUPPORT/${var} .endfor +# Enable additional options that default to NO in FreeBSD +__DEFAULT_YES_OPTIONS += \ + CLANG_EXTRAS \ + # Disable a bunch of additional options that default to yes in FreeBSD __DEFAULT_NO_OPTIONS += \ ATM \ -- 2.37.1 From 690d66a8d2595ca650d2e285d08e888c908602fb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 May 2015 17:53:19 +0200 Subject: [PATCH 071/310] etc/master.passwd: Change root's login shell to sh Now that csh is no longer compiled by default that seems like a rather swell idea. Obtained from: ElectroBSD --- etc/master.passwd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/etc/master.passwd b/etc/master.passwd index a1be886f1e8d..b61c13fdd9fe 100644 --- a/etc/master.passwd +++ b/etc/master.passwd @@ -1,6 +1,6 @@ # $FreeBSD$ # -root::0:0::0:0:Charlie &:/root:/bin/csh +root::0:0::0:0:Charlie &:/root:/bin/sh toor:*:0:0::0:0:Bourne-again Superuser:/root: daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin operator:*:2:5::0:0:System &:/:/usr/sbin/nologin -- 2.37.1 From 93a48429cf76075ab77547cf1faec3972bdb726a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 5 Jun 2015 12:03:46 +0200 Subject: [PATCH 072/310] release/Makefile: Set German keyboard map Obtained from: ElectroBSD --- release/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/release/Makefile b/release/Makefile index d12e799cbabc..46be7b49d4ea 100644 --- a/release/Makefile +++ b/release/Makefile @@ -196,6 +196,7 @@ disc1: packagesystem ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf echo sendmail_enable=\"NONE\" > ${.TARGET}/etc/rc.conf echo hostid_enable=\"NO\" >> ${.TARGET}/etc/rc.conf + echo keymap=\"de\" >> ${.TARGET}/etc/rc.conf echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf cp ${.CURDIR}/rc.local ${.TARGET}/etc -- 2.37.1 From 74d823a023874e8d2c306a24a7b9c0b86293ce57 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 5 Jun 2015 12:06:14 +0200 Subject: [PATCH 073/310] release/Makefile: Set hostname to ${VOLUME_LABEL} Obtained from: ElectroBSD --- release/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/release/Makefile b/release/Makefile index 46be7b49d4ea..bc3757df2ae0 100644 --- a/release/Makefile +++ b/release/Makefile @@ -197,6 +197,7 @@ disc1: packagesystem echo sendmail_enable=\"NONE\" > ${.TARGET}/etc/rc.conf echo hostid_enable=\"NO\" >> ${.TARGET}/etc/rc.conf echo keymap=\"de\" >> ${.TARGET}/etc/rc.conf + echo hostname=\"${VOLUME_LABEL}\" >> ${.TARGET}/etc/rc.conf echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf cp ${.CURDIR}/rc.local ${.TARGET}/etc -- 2.37.1 From f589cea45eedaf2df5f7b1d1b5f6f8c8142cc254 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 15 Jan 2017 11:38:02 +0100 Subject: [PATCH 074/310] release/Makefile: Use the OSRELEASE verbatim as VOLUME_LABEL ... but shorten if the label would be too long otherwise. On ElectroBSD the OSRELEASE only contains useful information. Obtained from: ElectroBSD --- release/Makefile | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/release/Makefile b/release/Makefile index bc3757df2ae0..1daf627d5a9b 100644 --- a/release/Makefile +++ b/release/Makefile @@ -65,11 +65,13 @@ ${_V}!= eval $$(awk '/^${_V}=/{print}' ${.CURDIR}/../sys/conf/newvers.sh); echo .for _V in ${TARGET_ARCH} .if !empty(TARGET:M${_V}) OSRELEASE= ${TYPE}-${REVISION}-${BRANCH}-${TARGET} -VOLUME_LABEL= ${REVISION:C/[.-]/_/g}_${BRANCH:C/[.-]/_/g}_${TARGET} .else OSRELEASE= ${TYPE}-${REVISION}-${BRANCH}-${TARGET}-${TARGET_ARCH} -VOLUME_LABEL= ${REVISION:C/[.-]/_/g}_${BRANCH:C/[.-]/_/g}_${TARGET_ARCH} .endif +# The label length needs to be below 32 characters to fit into +# the UFS superblock. PRERELEASE is the only branch name that +# can prevent this, so we shorten it. +VOLUME_LABEL= ${OSRELEASE:C/PRERELEASE/PREREL/} .endfor .endif -- 2.37.1 From 7d4e1ddb8f1187b8f76641717f78f1d1965a6899 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 10 Jun 2015 12:29:37 +0200 Subject: [PATCH 075/310] release/Makefile: Add ${DIST_TARBALL_DIR} ... which contains the distribution tarball directory on the install media. Obtained from: ElectroBSD --- release/Makefile | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/release/Makefile b/release/Makefile index 1daf627d5a9b..b263d33744db 100644 --- a/release/Makefile +++ b/release/Makefile @@ -57,6 +57,9 @@ DISTDIR= dist NO_ROOT=1 .export NO_ROOT +# Path to the distribution tarballs on the created installation media. +DIST_TARBALL_DIR=usr/freebsd-dist + # Define OSRELEASE by using newvars.sh .if !defined(OSRELEASE) || empty(OSRELEASE) .for _V in TYPE BRANCH REVISION @@ -190,9 +193,9 @@ disc1: packagesystem MK_KERNEL_SYMBOLS=no MK_TESTS=no MK_DEBUG_FILES=no \ -DDB_FROM_SRC # Copy distfiles - mkdir -p ${.TARGET}/usr/freebsd-dist + mkdir -p ${.TARGET}/${DIST_TARBALL_DIR} for dist in MANIFEST $$(ls *.txz | grep -vE -- '(base|lib32)-dbg'); \ - do cp $${dist} ${.TARGET}/usr/freebsd-dist; \ + do cp $${dist} ${.TARGET}/${DIST_TARBALL_DIR}; \ done # Set up installation environment ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf @@ -217,8 +220,9 @@ bootonly: packagesystem MK_KERNEL_SYMBOLS=no MK_TESTS=no MK_DEBUG_FILES=no \ -DDB_FROM_SRC # Copy manifest only (no distfiles) to get checksums - mkdir -p ${.TARGET}/usr/freebsd-dist - cp MANIFEST ${.TARGET}/usr/freebsd-dist + mkdir -p ${.TARGET}/${DIST_TARBALL_DIR} + cp MANIFEST ${.TARGET}/${DIST_TARBALL_DIR} + # Set up installation environment ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf echo sendmail_enable=\"NONE\" > ${.TARGET}/etc/rc.conf @@ -235,9 +239,9 @@ dvd: packagesystem MK_TESTS=no MK_DEBUG_FILES=no \ -DDB_FROM_SRC # Copy distfiles - mkdir -p ${.TARGET}/usr/freebsd-dist + mkdir -p ${.TARGET}/${DIST_TARBALL_DIR} for dist in MANIFEST $$(ls *.txz | grep -v -- '(base|lib32)-dbg'); \ - do cp $${dist} ${.TARGET}/usr/freebsd-dist; \ + do cp $${dist} ${.TARGET}/${DIST_TARBALL_DIR}; \ done # Set up installation environment ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf -- 2.37.1 From efb5af7bf96a48a0a0ef589127ab4fdc18c8d90f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 10 Jun 2015 12:36:37 +0200 Subject: [PATCH 076/310] release/Makefile: ElectroBSDify the shiny new ${DIST_TARBALL_DIR} Obtained from: ElectroBSD --- release/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/Makefile b/release/Makefile index b263d33744db..3ce47e65b252 100644 --- a/release/Makefile +++ b/release/Makefile @@ -58,7 +58,7 @@ NO_ROOT=1 .export NO_ROOT # Path to the distribution tarballs on the created installation media. -DIST_TARBALL_DIR=usr/freebsd-dist +DIST_TARBALL_DIR=usr/electrobsd-dist # Define OSRELEASE by using newvars.sh .if !defined(OSRELEASE) || empty(OSRELEASE) -- 2.37.1 From bce45a83a869e5859f4559d40b6e382cfe16c643 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 15 Jun 2015 12:57:51 +0200 Subject: [PATCH 077/310] Add rc.d script to optionally automatically enable soft-protection when booting Obtained from: ElectroBSD --- libexec/rc/rc.d/Makefile | 1 + libexec/rc/rc.d/soft-protection | 46 +++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+) create mode 100755 libexec/rc/rc.d/soft-protection diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index 34846411dce9..cb0e10a9b565 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -99,6 +99,7 @@ CONFS= DAEMON \ securelevel \ serial \ sppp \ + soft-protection \ statd \ static_arp \ static_ndp \ diff --git a/libexec/rc/rc.d/soft-protection b/libexec/rc/rc.d/soft-protection new file mode 100755 index 000000000000..58bc02a0c037 --- /dev/null +++ b/libexec/rc/rc.d/soft-protection @@ -0,0 +1,46 @@ +#!/bin/sh +# +########################################################################### +# +# soft-protection - Enables cloudiatr soft protection on boot +# +########################################################################### +# +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ALL YOUR +# DATA IS BELONG TO THE SOFTWARE AND MAY BE EATEN BY IT. IF THAT IS NOT +# ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. +########################################################################### + +# PROVIDE: soft_protection +# REQUIRE: FILESYSTEMS + +. /etc/rc.subr + +name="soft_protection" +rcvar="soft_protection_enable" + +soft_protection_enable="${soft_protection_enable-NO}" + +start_cmd="enable_soft_protection" +stop_cmd=":" + +enable_soft_protection() { + # We don't use the soft-protect subcommand + # because it may require user feedback. + cloudiatr cmd cloudiatr_soft_protect +} + +load_rc_config "${name}" +run_rc_command "${1}" -- 2.37.1 From be22cafe4092cca1b77c26d8e80d5392288120c4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 23 Jul 2015 19:16:44 +0200 Subject: [PATCH 078/310] dtrace_consume(): Warn if cpu cores aren't completely in sync ... instead of aborting. While the results may be not completely accurate, in some cases it may not matter. Previously the assertion would sometimes be triggered on a systems with poor timecounters (TSC-low(-100) ACPI-fast(900) i8254(0) dummy(-1000000)). Obtained from: ElectroBSD --- .../opensolaris/lib/libdtrace/common/dt_consume.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c index 7311f5340a46..d5e8b93b9da7 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c @@ -30,6 +30,7 @@ #include #include +#include #include #include #include @@ -3023,7 +3024,13 @@ dtrace_consume(dtrace_hdl_t *dtp, FILE *fp, if (buf != NULL) { if (first_timestamp == 0) first_timestamp = buf->dtbd_timestamp; - assert(buf->dtbd_timestamp >= first_timestamp); + if (buf->dtbd_timestamp < first_timestamp) { + warnx("cpu clocks out of sync " + "(%ju < %ju; offset: %ju). " + "Results may be incorrect!", + buf->dtbd_timestamp, first_timestamp, + first_timestamp - buf->dtbd_timestamp); + } dt_pq_insert(dtp->dt_bufq, buf); drops[i] = buf->dtbd_drops; -- 2.37.1 From fef456803131c5126f5e17aaf41fa799a6b2c674 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 13 Oct 2015 18:19:42 +0200 Subject: [PATCH 079/310] libdtrace: Replace another timestamp related assert() with a warning (XXX: remove completely) It triggered a lot less often, but was still annoying. Example: 2015 Oct 13 18:01:06: v_free_target: 20888, v_free_count: 44001, v_cache_count: 0. 2015 Oct 13 18:01:06: v_inactive_target: 31332. v_inactive_count: 364311. deficit: 0 2015 Oct 13 18:01:06: after-inactive-scan pass 3: page shortage: -23113, inactive page surplus: 10000, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:06: before-active-scan pass 3: page_shortage: -356092. 2015 Oct 13 18:01:06: after-active-scan pass 3: page_shortage: -356092. 2015 Oct 13 18:01:06: checked-inactive-pages: Inactive page surplus: 10000, above limit: 252979. dtrace: cpu clocks out of sync (18128465664478 < 18128471798881; offset: 6134403). Results may be incorrect! dtrace: cpu clocks out of sync (18129463671844 < 18129469778266; offset: 6106422). Results may be incorrect! 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253446. 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253446. 2015 Oct 13 18:01:07: before-inactive-scan: Scan goal 1: Move inactive to cache or free. 2015 Oct 13 18:01:07: before-inactive-scan: Page shortage: -37096, inactive page surplus: 10000 2015 Oct 13 18:01:07: v_free_target: 20888, v_free_count: 57984, v_cache_count: 0. 2015 Oct 13 18:01:07: v_inactive_target: 31332. v_inactive_count: 364778. deficit: 0 2015 Oct 13 18:01:07: after-inactive-scan pass 1: page shortage: -37203, inactive page surplus: 9893, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253250. 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253250. 2015 Oct 13 18:01:07: before-inactive-scan: Scan goal 2: Launder dirty pages. 2015 Oct 13 18:01:07: before-inactive-scan: Page shortage: -37191, inactive page surplus: 10000 2015 Oct 13 18:01:07: v_free_target: 20888, v_free_count: 58079, v_cache_count: 0. 2015 Oct 13 18:01:07: v_inactive_target: 31332. v_inactive_count: 364582. deficit: 0 2015 Oct 13 18:01:07: after-inactive-scan pass 2: page shortage: -37194, inactive page surplus: 9997, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:07: before-active-scan pass 2: page_shortage: -370441. 2015 Oct 13 18:01:07: after-active-scan pass 2: page_shortage: -370441. 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253268. dtrace: cpu clocks out of sync (18130462652022 < 18130468761237; offset: 6109215). Results may be incorrect! 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253331. 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253331. 2015 Oct 13 18:01:08: before-inactive-scan: Scan goal 1: Move inactive to cache or free. 2015 Oct 13 18:01:08: before-inactive-scan: Page shortage: -34625, inactive page surplus: 10000 2015 Oct 13 18:01:08: v_free_target: 20888, v_free_count: 55513, v_cache_count: 0. 2015 Oct 13 18:01:08: v_inactive_target: 31332. v_inactive_count: 364663. deficit: 0 2015 Oct 13 18:01:08: after-inactive-scan pass 1: page shortage: -34645, inactive page surplus: 9980, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253286. 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253286. 2015 Oct 13 18:01:08: before-inactive-scan: Scan goal 2: Launder dirty pages. 2015 Oct 13 18:01:08: before-inactive-scan: Page shortage: -34395, inactive page surplus: 10000 2015 Oct 13 18:01:08: v_free_target: 20888, v_free_count: 55283, v_cache_count: 0. 2015 Oct 13 18:01:08: v_inactive_target: 31332. v_inactive_count: 364618. deficit: 0 2015 Oct 13 18:01:08: after-inactive-scan pass 2: page shortage: -34396, inactive page surplus: 9999, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:08: before-active-scan pass 2: page_shortage: -367391. 2015 Oct 13 18:01:08: after-active-scan pass 2: page_shortage: -367391. 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253285. 2015 Oct 13 18:01:09: checked-inactive-pages: Inactive page surplus: 10000, above limit: 256158. 2015 Oct 13 18:01:09: before-inactive-scan: Scan goal 3: Pageout dirty pages. 2015 Oct 13 18:01:09: before-inactive-scan: Page shortage: -34665, inactive page surplus: 10000 2015 Oct 13 18:01:09: v_free_target: 20888, v_free_count: 55553, v_cache_count: 0. 2015 Oct 13 18:01:09: v_inactive_target: 31332. v_inactive_count: 367490. deficit: 0 dtrace: cpu clocks out of sync (18131462608939 < 18131468746340; offset: 6137401). Results may be incorrect! Assertion failed: (timestamp >= dtp->dt_last_timestamp), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c, line 3352. Obtained from: ElectroBSD --- .../opensolaris/lib/libdtrace/common/dt_consume.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c index d5e8b93b9da7..ce8717dec593 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c @@ -3047,6 +3047,15 @@ dtrace_consume(dtrace_hdl_t *dtp, FILE *fp, break; timestamp = dt_buf_oldest(buf, dtp); + if (timestamp < dtp->dt_last_timestamp) { + warnx("cpu clocks out of sync " + "(%ju < %ju; offset: %ju). " + "Results may be incorrect!", + timestamp, dtp->dt_last_timestamp, + dtp->dt_last_timestamp - timestamp); + } + dtp->dt_last_timestamp = timestamp; + if (timestamp == buf->dtbd_timestamp) { /* * We've reached the end of the time covered -- 2.37.1 From 6c83c815d2443923b5c6d5ade3de97b0cdde4766 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 16 Jun 2015 15:21:46 +0200 Subject: [PATCH 080/310] share/dtrace: Import geli-key-monitor Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/README | 3 +- share/dtrace/geli-key-monitor | 191 ++++++++++++++++++++++++++++++++++ 3 files changed, 194 insertions(+), 1 deletion(-) create mode 100755 share/dtrace/geli-key-monitor diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 8ae7aad56fa5..e908fc4fb2e7 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -9,6 +9,7 @@ PACKAGE= dtrace SCRIPTS= blocking \ disklatency \ disklatencycmd \ + geli-key-monitor \ hotopen \ nfsattrstats \ nfsclienttime \ diff --git a/share/dtrace/README b/share/dtrace/README index 6855c6aee721..aeb2a06727b5 100644 --- a/share/dtrace/README +++ b/share/dtrace/README @@ -2,4 +2,5 @@ $FreeBSD$ This directory contains scripts for use with the DTrace system. These files and directories contain code generated by the FreeBSD -Project for use with DTrace on FreeBSD. +Project for use with DTrace on FreeBSD and code generated by the +ElectroBSD project for use with DTrace on ElectroBSD. diff --git a/share/dtrace/geli-key-monitor b/share/dtrace/geli-key-monitor new file mode 100755 index 000000000000..2cf4ba5c09be --- /dev/null +++ b/share/dtrace/geli-key-monitor @@ -0,0 +1,191 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * geli-key-monitor + * + * Traces GELI to print the beginning of various keys and warns + * about already known ones. For motivation and example output see: + * https://www.fabiankeil.de/gehacktes/geli-key-monitor/ + * + * Copyright (c) 2012 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +dtrace:::BEGIN +{ + algo[ 2] = "3DES-CBC"; + algo[ 3] = "Blowfish-CBC"; + algo[11] = "AES-CBC"; + algo[21] = "Camellia-CBC"; + algo[22] = "AES-XTS"; + + /* + * These are the first bytes of known weak keys generated with + * an unitialized sc->sc_ekey on a little-endian system. + */ + known_keys[0x06d800ef] = 1; + known_keys[0x252c6a6d] = 1; + known_keys[0xa72b9c7c] = 1; + known_keys[0xfde44683] = 1; + + G_ELI_FLAG_ENC_IVKEY = 0x00400000; + + g_eli_key_stats_available = 0; + g_eli_hold_stats_available = 0; + + /* + * Geli v5 and higher use a different key for each GB, + * so large disks have several thousand keys. + * Monitoring only the first few should do, keeps the + * output and reduces the likelihood of prevents dynvar + * drops. + */ + max_disk_keys_to_monitor = 4; + + printf("%Y: Monitoring geli keys (up to %d encryption keys per disk). %s\n", + walltimestamp, max_disk_keys_to_monitor, "Press CTRL-C to exit."); +} + +fbt::g_eli_mkey_propagate:entry +{ + self->sc = (struct g_eli_softc *)arg0; + self->mkey = (struct g_eli_key *)arg1; + + self->geom_name = stringof(self->sc->sc_geom->name); + self->monitored_keys = 0; + self->key_limit_reached = 0; + + this->algo = self->sc->sc_ealgo; + this->algo_name = (algo[this->algo] != NULL) ? algo[this->algo] : "Unregistered"; + this->geli_version = self->sc->sc_version; + + printf("%Y: %s: %s:%-6s: version: %d, algo: %s, flags: 0x%x (FLAG_ENC_IVKEY: %d) ", + walltimestamp, + self->geom_name, + probefunc, probename, + this->geli_version, this->algo_name, + self->sc->sc_flags, + (self->sc->sc_flags & G_ELI_FLAG_ENC_IVKEY) != 0 + ); + printf("mkey: %08.8x, sc_mkey: %08.8x, sc_ekey: %08.8x\n", + *(uint32_t *)(self->mkey), + *(uint32_t *)(self->sc->sc_mkey), + *(uint32_t *)(self->sc->sc_ekey)); +} + +fbt::g_eli_mkey_propagate:return +/self->sc != NULL/ +{ + printf("%Y: %s: %s:%-6s: mkey: %08.8x, sc_mkey: %08.8x, sc_ekey: %08.8x, sc_ekeys_allocated: %d\n", + walltimestamp, + self->geom_name, + probefunc, probename, + *(uint32_t *)(self->mkey), + *(uint32_t *)(self->sc->sc_mkey), + *(uint32_t *)(self->sc->sc_ekey), + self->sc->sc_ekeys_allocated); +} + +fbt::g_eli_key_fill:entry +{ + self->sc = (struct g_eli_softc *)arg0; + self->key = (struct g_eli_key *)arg1; + self->geom_name = stringof(self->sc->sc_geom->name); +} + +fbt::g_eli_key_fill: +/(self->sc != NULL) && !self->key_limit_reached/ +{ + this->key = *(uint32_t *)(self->key->gek_key); + printf("%Y: %s: %s:%-6s: key->gek_key: %8.8x (%d).%s\n", + walltimestamp, + self->geom_name, + probefunc, probename, + this->key, + self->monitored_keys, + known_keys[this->key] ? " Key looks familiar!" : ""); +} + +fbt::g_eli_key_fill:return +/(self->sc != NULL) && !self->key_limit_reached/ +{ + this->key = *(uint32_t *)(self->key->gek_key); + @g_eli_keys[self->monitored_keys, this->key, self->geom_name] = count(); + /* + * Register the generated key as known. + * Unless the provider is reattached we do not want to see it again. + */ + known_keys[this->key] = 1; + g_eli_key_stats_available = 1; + self->monitored_keys++; +} + +fbt::g_eli_key_fill:return +/(self->sc != NULL) && (self->sc->sc_ekeys_allocated == max_disk_keys_to_monitor - 1)/ +{ + printf("%Y: %s: %s:%-6s: Encryption key limit per disk reached.\n", + walltimestamp, self->geom_name, probefunc, probename); + self->key_limit_reached = 1; +} + +fbt::g_eli_key_hold:entry +{ + self->sc = (struct g_eli_softc *)arg0; + self->offset = (off_t)arg1; + self->blocksize = (size_t)arg2; + self->geom_name = stringof(self->sc->sc_geom->name); + + this->first_mkey_bytes = *(uint32_t *)(self->sc->sc_mkey); + this->first_ekey_bytes = *(uint32_t *)(self->sc->sc_ekey); + /* + * We only check the first bytes here, so false-positives are + * theoretically posible although unlikely. + */ + this->empty_ekey = (0 == this->first_ekey_bytes); + + this->geli_version = self->sc->sc_version; + this->algo = self->sc->sc_ealgo; + this->algo_name = (algo[this->algo] != NULL) ? algo[this->algo] : "Unregistered"; + + @g_eli_key_hold[self->geom_name, + this->geli_version, + this->algo_name, + this->algo, + this->first_mkey_bytes, + this->first_ekey_bytes] = count(); + + g_eli_hold_stats_available = 1; +} + +tick-60sec, +dtrace:::END +/g_eli_hold_stats_available/ +{ + printf("\n---------------------------------------------------------\n"); + printf("%Y: g_eli_key_hold() calls so far:\n", walltimestamp); + printf("%-20s %12s %18s %11s %11s %10s\n", + "Provider", "Geli version", "Algorithm", "mkey start", "ekey start", "calls"); + printa("%-20s %12d %13s (%2d) %08x %08x %@10d\n", @g_eli_key_hold); +} + +tick-60sec, +dtrace:::END +/g_eli_key_stats_available/ +{ + printf("\n%Y: g_eli_key count (only works for geli version 5 or higher):\n", walltimestamp); + printa("%@u #%04d %8.8x %-25s\n", @g_eli_keys); +} -- 2.37.1 From 3dfaff4175e4ee257b0057d679afedd65cca5d91 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 4 Sep 2015 20:07:22 +0200 Subject: [PATCH 081/310] share/dtrace: Import fbt-time ... which llquantizes the time spent in a given kernel function Among other things this can be useful for general system tuning and analysis of lock contention (once you know where to expect it). Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/fbt-time | 80 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 81 insertions(+) create mode 100755 share/dtrace/fbt-time diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index e908fc4fb2e7..80cbc5038e83 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -9,6 +9,7 @@ PACKAGE= dtrace SCRIPTS= blocking \ disklatency \ disklatencycmd \ + fbt-time \ geli-key-monitor \ hotopen \ nfsattrstats \ diff --git a/share/dtrace/fbt-time b/share/dtrace/fbt-time new file mode 100755 index 000000000000..27a950b63a0c --- /dev/null +++ b/share/dtrace/fbt-time @@ -0,0 +1,80 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * fbt-time + * + * Measures the time spent in a given kernel function and llquantizes + * the results for the traced calls. Example output can be seen at: + * http://www.fabiankeil.de/gehacktes/electrobsd/zfs-arc-tuning/ + * + * WARNING: Tracing kernel functions that get executed several thousand + * times per second may negatively impact system performance + * and can even cause brief lockups. Before using this script + * on production systems you may want to sample the traced + * kernel function first. + * + * XXX: This script should be fleshed out to be more resilient to + * incorrect input and provide more awesome stats. A better name + * would be nice, too. + * + * Copyright (c) 2014 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +dtrace:::BEGIN +{ + traced_function = $$1; + + multiple = "microseconds"; + divisor = 1000; + executions = 0; + + this->timestamp = walltimestamp; + this->msecs = (this->timestamp / 1000000) % 1000; + printf("%Y.%.3d: Tracing '%s'. Hit CTRL-C to exit.\n", + this->timestamp, this->msecs, traced_function); +} + +fbt::$$1:entry +{ + self->timestamp = timestamp; + executions++; +} + +fbt::$$1:return +/self->timestamp/ +{ + this->elapsed = (timestamp - self->timestamp) / divisor; + @elapsed_time = llquantize(this->elapsed, 10, 0, 5, 10); + @elapsed_avg["Average"] = avg(this->elapsed); + self->elapsed = 0; +} + +tick-10sec, +END +/executions/ +{ + printf("%Y: Runtime for %s() in %s\n", walltimestamp, traced_function, multiple); + printa(@elapsed_avg); + printa(@elapsed_time); +} + +END +/executions == 0/ +{ + printf("%Y: Looks like %s() wasn't called yet.\n", walltimestamp, traced_function); +} -- 2.37.1 From aee4f48a7f10625a016d7c9c20bbe9b09370503e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 15 May 2016 12:09:26 +0200 Subject: [PATCH 082/310] share/dtrace: Add geli-request-monitor ... which shows the BIO commands GELI received, their bio lengths from the caller's point of view and the maximum number of commands in flight. Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/geli-request-monitor | 91 +++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100755 share/dtrace/geli-request-monitor diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 80cbc5038e83..99f471d9527e 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -11,6 +11,7 @@ SCRIPTS= blocking \ disklatencycmd \ fbt-time \ geli-key-monitor \ + geli-request-monitor \ hotopen \ nfsattrstats \ nfsclienttime \ diff --git a/share/dtrace/geli-request-monitor b/share/dtrace/geli-request-monitor new file mode 100755 index 000000000000..70336060057d --- /dev/null +++ b/share/dtrace/geli-request-monitor @@ -0,0 +1,91 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * geli-request-monitor + * + * Shows the length distribution of various GELI commands from + * the callers point of view (that is, without geli overhead). + * + * Copyright (c) 2016-2017 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet + +dtrace:::BEGIN +{ + bio_command[0x01] = "BIO_READ"; + bio_command[0x02] = "BIO_WRITE"; + bio_command[0x03] = "BIO_DELETE"; + bio_command[0x04] = "BIO_GETATTR"; + bio_command[0x05] = "BIO_FLUSH"; + bio_command[0x06] = "BIO_CMD0"; + bio_command[0x07] = "BIO_CMD1"; + bio_command[0x08] = "BIO_CMD2"; + bio_command[0x09] = "BIO_ZONE"; +} + +fbt::g_eli_auth_run:entry, +fbt::g_eli_crypto_run:entry +{ + this->bp = (struct bio *)arg1; + this->sc = (struct g_eli_softc *)this->bp->bio_to->geom->softc; + + this->geom_name = stringof(this->sc->sc_geom->name); + this->length = this->bp->bio_length; + this->inflight = this->sc->sc_inflight; + + this->bio_command = (bio_command[this->bp->bio_cmd] != NULL) ? + bio_command[this->bp->bio_cmd] : "Unregistered command"; + + @inflight_max[this->geom_name] = max(this->inflight); + @inflight_max_total[this->geom_name] = max(this->inflight); + + @length[this->geom_name, this->bio_command] = + lquantize(this->length, 4096, 1052672, 4096); + @length_total[this->geom_name, this->bio_command] = + lquantize(this->length, 4096, 1052672, 4096); +/* + printf("%s: Length: %d, Inflight: %d", this->bio_command, + this->length, this->inflight); +*/ + + stats_available = 1; +} + +tick-60s +/stats_available/ +{ + printf("------------------------------\n"); + printf("Max requests inflight:\n"); + printa(@inflight_max); + printf("\nbp->bio_length:\n"); + printa(@length); + trunc(@length); + trunc(@inflight_max); + stats_available = 0; +} + +END +{ + /* Explicitly ditch interval stats so they aren't printed */ + trunc(@length); + trunc(@inflight_max); + + printf("\n------------------------------\n"); + printf("Max requests inflight:\n"); + printa(@inflight_max_total); + printf("\nbp->bio_length:\n"); + printa(@length_total); +} -- 2.37.1 From 90c25c7c0c8ade1c282e428e49590ba0b216dd26 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 20 Sep 2015 19:10:51 +0200 Subject: [PATCH 083/310] share/dtrace: Add monitor-page-scanner ... which, who would have guessed it, monitors the vm page scanner. It's useful to tune the sysctls for the inactive page limit. Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/monitor-page-scanner | 168 ++++++++++++++++++++++++++++++ 2 files changed, 169 insertions(+) create mode 100755 share/dtrace/monitor-page-scanner diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 99f471d9527e..b58298ffbfb7 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -13,6 +13,7 @@ SCRIPTS= blocking \ geli-key-monitor \ geli-request-monitor \ hotopen \ + monitor-page-scanner \ nfsattrstats \ nfsclienttime \ siftr \ diff --git a/share/dtrace/monitor-page-scanner b/share/dtrace/monitor-page-scanner new file mode 100755 index 000000000000..85537caff8bf --- /dev/null +++ b/share/dtrace/monitor-page-scanner @@ -0,0 +1,168 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * monitor-page-scanner + * + * Traces the vm page scanner. + * + * Relies on SDT probes that currrently are not part of vanilla FreeBSD. + * + * Copyright (c) 2015 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +dtrace:::BEGIN +{ + goal[0] = "Update active LRU/deactivate pages"; + goal[1] = "Move inactive to cache or free"; + goal[2] = "Launder dirty pages"; + goal[3] = "Pageout dirty pages"; + start_time = walltimestamp; + + min_pass = 2; + + printf("%Y: Monitoring the page scanner. Minimum pass value to show 'boring' scans ", walltimestamp); + printf("without memory pressure or inactive page surplus: %d (%s). Press CTRL-C to abort.\n", + min_pass, goal[min_pass]); +} + +vm:kernel::checked-inactive-pages +{ + this->to_free = (int)arg0; +} + +vm:kernel::checked-inactive-pages +/this->to_free/ +{ + this->above_limit = (int)arg1; + + printf("%Y: %s: Inactive page surplus: %d, above limit: %d.\n", + walltimestamp, probename, this->to_free, this->above_limit); +} + +vm:kernel::before-inactive-scan +{ + this->pass = (int)arg1; + this->page_shortage = (int)arg2; + this->inactive_page_surplus = (int)arg4; + + /* + * Every pass code above 3 is treated like 3, + * adjust index accordingly. + */ + this->goal_index = (this->pass <= 3) ? this->pass : 3; + this->goal = goal[this->goal_index]; + @goals[this->goal] = count(); + @goals_total[this->goal] = count(); + last_pass[this->pass] = timestamp; +} + +vm:kernel::before-inactive-scan +/(this->pass >= min_pass) || (this->page_shortage > 0) +|| (this->inactive_page_surplus > 0)/ +{ + /* this->vmd = (struct vm_domain *)arg0; */ + this->deficit = (int)arg3; + + printf("%Y: %s: Scan goal %d: %s.\n", + walltimestamp, probename, this->pass, this->goal); + printf("%Y: %s: Page shortage: %d, inactive page surplus: %d\n", + walltimestamp, probename, this->page_shortage, this->inactive_page_surplus); + printf("%Y: v_free_target: %d, v_free_count: %d, v_cache_count: %d.\n", + walltimestamp, `vm_cnt.v_free_target, `vm_cnt.v_free_count, `vm_cnt.v_cache_count); + printf("%Y: v_inactive_target: %d. v_inactive_count: %d. deficit: %d\n", + walltimestamp, `vm_cnt.v_inactive_target, `vm_cnt.v_inactive_count, this->deficit); +} + +vm:kernel::after-inactive-scan +{ + this->pass = (int)arg1; + this->page_shortage = (int)arg2; + this->addl_page_shortage = (int)arg3; + this->vnodes_skipped = (int)arg4; + this->inactive_page_surplus = (int)arg5; +} + +vm:kernel::after-inactive-scan +/(this->pass >= min_pass) || (this->page_shortage > 0) || +(this->addl_page_shortage > 0) || (this->vnodes_skipped > 0) || +(this->inactive_page_surplus > 0)/ +{ + printf("%Y: %s pass %d: page shortage: %d, inactive page surplus: %d, addl shortage: %d, vnodes skipped: %d.\n", + walltimestamp, probename, this->pass, this->page_shortage, + this->inactive_page_surplus, this->addl_page_shortage, this->vnodes_skipped); +} + +vm:kernel::before-active-scan, +vm:kernel::after-active-scan +{ + this->pass = (int)arg1; + this->page_shortage = (int)arg2; +} + +vm:kernel::before-active-scan, +vm:kernel::after-active-scan +/(this->pass >= min_pass) || (this->page_shortage > 0)/ +{ + printf("%Y: %s pass %d: page_shortage: %d.\n", + walltimestamp, probename, this->pass, this->page_shortage); +} + +tick-60s +{ + printf("%Y: Scan goals in the previous minute:", walltimestamp); + printa(@goals); + trunc(@goals); +} + +tick-60s +/last_pass[1]/ +{ + this->pass = 1; + this->elapsed = (timestamp - last_pass[this->pass]) / 1000000000; + printf("%Y: Seconds since last '%s' pass: %d.\n", + walltimestamp, goal[this->pass], this->elapsed); +} + +tick-60s +/last_pass[2]/ +{ + this->pass = 2; + this->elapsed = (timestamp - last_pass[this->pass]) / 1000000000; + printf("%Y: Seconds since last '%s' pass: %d.\n", + walltimestamp, goal[this->pass], this->elapsed); +} + +tick-60s +/last_pass[3]/ +{ + this->pass = 3; + this->elapsed = (timestamp - last_pass[this->pass]) / 1000000000; + printf("%Y: Seconds since last '%s' pass: %d.\n", + walltimestamp, goal[this->pass], this->elapsed); +} + +END +{ + printf("%Y: Scan goals since start of script at %Y:", + walltimestamp, start_time); + printa(@goals_total); + + /* Clear aggregates so DTrace does not show them again. */ + trunc(@goals_total); + trunc(@goals); +} -- 2.37.1 From ff7daf92b66aa6344de5cd3ef5fc60add48ceb11 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 22 Apr 2017 13:41:41 +0200 Subject: [PATCH 084/310] share/dtrace: Add zfsdbg-msg ... which shows ZFS debug messages. Obtained from: ElectroBSD --- share/dtrace/Makefile | 3 ++- share/dtrace/zfsdbg-msg | 12 ++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100755 share/dtrace/zfsdbg-msg diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index b58298ffbfb7..6efaf27d1304 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -21,7 +21,8 @@ SCRIPTS= blocking \ tcpdebug \ tcpstate \ tcptrack \ - udptrack + udptrack \ + zfsdbg-msg SCRIPTSDIR= ${SHAREDIR}/dtrace diff --git a/share/dtrace/zfsdbg-msg b/share/dtrace/zfsdbg-msg new file mode 100755 index 000000000000..80e65b2dbae5 --- /dev/null +++ b/share/dtrace/zfsdbg-msg @@ -0,0 +1,12 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * zfs-dbgmsg - Emit ZFS debug messages + ***************************************************************************/ + +#pragma D option quiet + +zfs-dbgmsg +{ + printf("%Y: %s\n", walltimestamp, stringof(arg0)) +} -- 2.37.1 From 024f63d283eb4a05aceb575bb80bce453cb4d07b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 30 Sep 2015 21:15:20 +0200 Subject: [PATCH 085/310] Add two DTrace scripts. WIP Obtained from: ElectroBSD --- share/dtrace/io-breakdown | 19 +++++++++++++++++++ share/dtrace/open-ios | 31 +++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+) create mode 100755 share/dtrace/io-breakdown create mode 100755 share/dtrace/open-ios diff --git a/share/dtrace/io-breakdown b/share/dtrace/io-breakdown new file mode 100755 index 000000000000..ee9e592f1bb5 --- /dev/null +++ b/share/dtrace/io-breakdown @@ -0,0 +1,19 @@ +#!/usr/sbin/dtrace -s + +/* This is the example from dtrace_io(4) */ + +#pragma D option quiet +#pragma D option dynvarsize=16m + +#pragma D option quiet + +io:::start +{ + @[args[1]->device_name, execname, pid] = sum(args[0]->bio_bcount); +} + +END +{ + printf("%10s %20s %10s %15s", "DEVICE", "APP", "PID", "BYTES"); + printa("%10s %20s %10d %15@d", @); +} diff --git a/share/dtrace/open-ios b/share/dtrace/open-ios new file mode 100755 index 000000000000..ee8bbe0d922d --- /dev/null +++ b/share/dtrace/open-ios @@ -0,0 +1,31 @@ +#!/usr/sbin/dtrace -s + +#pragma D option quiet +#pragma D option dynvarsize=16m + +dtrace:::BEGIN +{ + printf("Tracing... Hit Ctrl-C to end.\n"); +} + +io:::start +{ + printf("%Y: Device name\n", walltimestamp); + print(args[1]->device_name); + start_time[arg0] = timestamp; + @open_ios = count(); +} + +io:::done +/this->start = start_time[arg0]/ +{ + @done_ios = count(); +} + +tick-10s +{ + printf("%Y: Started ios", walltimestamp); + printa(@open_ios); + printf("%Y: Done ios", walltimestamp); + printa(@done_ios); +} -- 2.37.1 From 31f1fa5d8d85bb7fe6035f677e273efd7a7166cb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 25 Jun 2015 16:46:35 +0200 Subject: [PATCH 086/310] release/Makefile: Note that the release process may silently fail Obtained from: ElectroBSD --- release/Makefile | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/release/Makefile b/release/Makefile index 3ce47e65b252..c165c3dc527a 100644 --- a/release/Makefile +++ b/release/Makefile @@ -2,6 +2,14 @@ # # Makefile for building releases and release media. # +# Note that the release process is rather fragile and lots +# of variables depend on each other in mysterious and +# undocumented ways. +# +# If you don't get the magic right you should consider yourself +# lucky if the build fails, the usual outcome is a release that +# does not work. +# # User-driven targets: # cdrom: Builds release CD-ROM media (disc1.iso) # dvdrom: Builds release DVD-ROM media (dvd1.iso) -- 2.37.1 From 3d49b7951b660266602d6076fd9135842c98bcd7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 25 Jun 2015 16:50:54 +0200 Subject: [PATCH 087/310] release/Makefile: Remove more stuff that does not apply to ElecroBSD from the disc1 target Obtained from: ElectroBSD --- release/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/release/Makefile b/release/Makefile index c165c3dc527a..a315798ec66d 100644 --- a/release/Makefile +++ b/release/Makefile @@ -206,7 +206,6 @@ disc1: packagesystem do cp $${dist} ${.TARGET}/${DIST_TARBALL_DIR}; \ done # Set up installation environment - ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf echo sendmail_enable=\"NONE\" > ${.TARGET}/etc/rc.conf echo hostid_enable=\"NO\" >> ${.TARGET}/etc/rc.conf echo keymap=\"de\" >> ${.TARGET}/etc/rc.conf -- 2.37.1 From 73a240c40982b09e8457eb780a097e9f0d84552e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 25 Jun 2015 17:08:38 +0200 Subject: [PATCH 088/310] Makefile.inc1: Workaround passwd and friends missing in the base.txz Obtained from: ElectroBSD --- Makefile.inc1 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Makefile.inc1 b/Makefile.inc1 index c4a7b38b4cbe..16a82f93b116 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1470,6 +1470,9 @@ distributeworld installworld stageworld: _installcheck_world .PHONY -e 's@(type=)@uid=0 gid=0 time=${EPOCH_DATE}.0 \1@' \ -e 's@//@/@g' -i '.bak' ${METALOG} .endif + @# Workaround for parts of etc mysteriously not being added below base. + @# As usual the location in dist is fine. + sed -e 's@^\./etc@./base/etc@' -i '.etc.bak' ${METALOG} .for dist in base ${EXTRA_DISTRIBUTIONS} @# For each file that exists in this dist, print the corresponding @# line from the METALOG. This relies on the fact that -- 2.37.1 From d15fe795a0a72f8d6a8fd28622e7f9053e7d6f14 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 3 Jul 2015 09:50:52 +0200 Subject: [PATCH 089/310] copyright.h: Use more insightful trademark information Obtained from: ElectroBSD --- sys/sys/copyright.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/sys/copyright.h b/sys/sys/copyright.h index 388e123b5702..c024210ce3fb 100644 --- a/sys/sys/copyright.h +++ b/sys/sys/copyright.h @@ -37,7 +37,7 @@ /* Foundation */ #define TRADEMARK_Foundation \ - "FreeBSD is a registered trademark of The FreeBSD Foundation.\n" + "ElectroBSD ain't no registered trademark of The ElectroBSD Foundation (which does not exist).\n" /* Berkeley */ #define COPYRIGHT_UCB \ -- 2.37.1 From 17dbef8b41c69fa5000b21bede48d201eed9f02d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 8 Jul 2015 17:43:31 +0200 Subject: [PATCH 090/310] sys/kern: Remove another FreeBSD reference from the boot messages Obtained from: ElectroBSD --- sys/kern/subr_smp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/subr_smp.c b/sys/kern/subr_smp.c index d66120666f9d..4a6518f722e4 100644 --- a/sys/kern/subr_smp.c +++ b/sys/kern/subr_smp.c @@ -169,7 +169,7 @@ mp_start(void *dummy) } cpu_mp_start(); - printf("FreeBSD/SMP: Multiprocessor System Detected: %d CPUs\n", + printf("SMP: Multiprocessor System Detected: %d CPUs\n", mp_ncpus); /* Provide a default for most architectures that don't have SMT/HTT. */ -- 2.37.1 From 1a1bcd0959cf997233a3dac0bac87d006c4e5ebd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 12 Jul 2015 13:11:28 +0200 Subject: [PATCH 091/310] tools/test/devrandom: Import arctest d9a5fc80, a wrapper around dieharder Obtained from: ElectroBSD --- tools/test/devrandom/arc4test | 278 ++++++++++++++++++++++++++++++++++ 1 file changed, 278 insertions(+) create mode 100755 tools/test/devrandom/arc4test diff --git a/tools/test/devrandom/arc4test b/tools/test/devrandom/arc4test new file mode 100755 index 000000000000..c824997a4a38 --- /dev/null +++ b/tools/test/devrandom/arc4test @@ -0,0 +1,278 @@ +#!/bin/sh + +############################################################################ +# arc4test +# +# Collects "entropy" and lets dieharder analyze it later on. The collected +# entropy is split into smaller files so the data collected in multiple +# runs can be easily interleaved and tested together. +# +# By default, entropy files are generated with arc4cat, a wrapper around +# arc4random_buf(3) which is suspected of "not returning very random data" +# between FreeBSD r273872 and r278907. +# +# So far it looks like the data may be "random enough" to pass the tests. +# +# Usage: +# arc4test build : Build arc4cat in $ARC4CAT_DIR +# arc4test collect : Collect potential entropy with arc4cat +# arc4test collect -d : Collect potential entropy with Dilbert PNRG +# arc4test analyze : Interleave collected entropy files and +# pipe them into dieharder. +# arc4test analyze -f : Try to spead up things by caching the interleaved +# entropy in a single file. Reuses the file if it +# already exists. +# arc4test remix : (Re)build an entropy cache file based on the +# previously collected entropy files. Roughfly +# doubles the required disk space but may significantly +# improve performance. +# arc4test cat : Dump interleaved entropy files to stdout +# +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +############################################################################ + +ARC4CAT_DIR=. +ARC4CAT="${ARC4CAT_DIR}/arc4cat" +# May not contain spaces etc. +ENTROPY_DIR="./entropy" +ENTROPY_SUBDIR_PREFIX="collection-" +# This is currently the block size for dd, setting it "too high" +# is not expected to work, however that's not good for input mixing +# later on anyway. +# +# Currently the entropy is split while it's being collected which is +# convenient from a programming point of view, but if the file size +# is small the interleave performance "may" (vulgo: will) suck. +ENTROPY_FILE_SIZE=4k +# Number of entropy files collected per run. If you increase ENTROPY_FILE_SIZE +# (or intend to do lots of collection runs) you may want to decrease this. +MAX_ENTROPY_FILES=100000 +ENTROPY_CACHE_FILE="${ENTROPY_DIR}/cached-entropy" + +prepare() { +} + +# The default entropy source +get_arc4cat_entropy() { + "${ARC4CAT}" +} + +# This is the reverse engineered PRNG from Dilbert strip 2001-10-25: +# http://dilbert.com/strip/2001-10-25 +# +# It is used instead of get_arc4cat_entropy() if the +# collect flag -d is set. +# +# According to the literature (see URL above) you can never be sure +# if the output is random, however the generator seems to fail all +# the dieharder tests and thus doesn't look nearly as good as Yarrow +# and Fortuna. +# +# Until this changes it will not be considered for ElectroBSD. +get_dilbert_entropy() { + while true; do + # The loop has been partially unrolled + # for increased performance! + echo -n "999999" + done +} + +get_shiny_new_entropy() { + local entropy_flag="${1}" + + if [ "${entropy_flag}" = "-d" ]; then + get_dilbert_entropy + else + get_arc4cat_entropy + fi +} + +collect_entropy() { + local entropy_flag \ + i entropy_file entropy_subdir + + entropy_flag="${1}" + i=0 + entropy_subdir="${ENTROPY_DIR}/${ENTROPY_SUBDIR_PREFIX}$(date +%s)" + + mkdir -p "${entropy_subdir}" + + ENTROPY_FILE_POSTFIX="" + + echo "Collecting ${MAX_ENTROPY_FILES} entropy files of size ${ENTROPY_FILE_SIZE} ..." + + # We don't call get_shiny_new_entropy() inside the loop as it + # would result in bits of entropy getting dropped on the floor + # between files. While we don't care about the "waste", we do + # care about not being able to test those bits later on. + get_shiny_new_entropy "${entropy_flag}" | while [ "${i}" -lt "${MAX_ENTROPY_FILES}" ]; do + entropy_file="$(printf "${entropy_subdir}/%.6i" "${i}")" + #echo "Creating ${entropy_file}" + dd bs="${ENTROPY_FILE_SIZE}" count=1 of="${entropy_file}" 2>/dev/null + i=$((i + 1)) + done +} + +create_entropy_cache() { + local \ + entropy_file + + entropy_file="${ENTROPY_DIR}/${LARGE_ENTROPY_FILE_NAME}" + + echo "Building a single entropy file '${ENTROPY_CACHE_FILE}' based on the files collected previously ..." 1>&2 + cat_collected_entropy > "${ENTROPY_CACHE_FILE}" +} + +replay_entropy() { + local fast_flag \ + entropy_file + + fast_flag="${1}" + + if [ "${fast_flag}" = "-f" ]; then + entropy_file="${ENTROPY_DIR}/${LARGE_ENTROPY_FILE_NAME}" + + if [ ! -f "${ENTROPY_CACHE_FILE}" ]; then + create_entropy_cache + fi + cat_entropy_cache + else + cat_collected_entropy + fi +} + +warn_about_entropy_reuse() { + echo "$0: Oh noes, we're out of collected entropy. Going back to the beginning." 1>&2 + echo "This shouldn't be a problem as long as no single test sees repeated data." 1>&2 +} + +cat_entropy_cache() { + while true; do + cat "${ENTROPY_CACHE_FILE}" + warn_about_entropy_reuse + done +} + +cat_collected_entropy() { + local \ + i f entropy_collections entropy_subdir entropy_file + + # XXX: Too fucking slow + #entropy_collections="$(find "${ENTROPY_DIR}/" -name "${ENTROPY_SUBDIR_PREFIX}*" -depth 1 -type 1)" + + # Not best practice but at least the performance doesn't suck + # and it works as expected. + + entropy_collections="${ENTROPY_DIR}/${ENTROPY_SUBDIR_PREFIX}"* + i=0 + while true; do + f="$(printf "%.6i" "${i}")" + for entropy_subdir in $entropy_collections; do + entropy_file="${entropy_subdir}/${f}" + #echo "Catting ${entropy_file}" + cat "${entropy_file}" || return 1 + done + i=$((i + 1)) + if [ "${i}" -eq "${MAX_ENTROPY_FILES}" ]; then + warn_about_entropy_reuse + i=0 + fi + done +} + +get_dieharder_tests() { + dieharder -l | awk '/-d/ {print $2}' +} + +analyze_collected_entropy() { + local fast_flag \ + test_number + + fast_flag="${1}" + + # We call replay_entropy() inside the loop to make sure + # the beginning of the collected entropy is checked by all tests + # (instead of having each test start at different offsets). + for test_number in $(get_dieharder_tests); do + replay_entropy ${fast_flag} | dieharder -g 200 -d "${test_number}" + done +} + +get_arc4cat_code() { + cat< +#include +#include + +int main(void) { + char buf[4096]; + + while (1) { + arc4random_buf(buf, sizeof(buf)); + write(1, buf, sizeof(buf)); + } +} +EOF +} + +build_arc4cat() { + mkdir -p "${ARC4CAT_DIR}" + cd "${ARC4CAT_DIR}" + get_arc4cat_code > arc4cat.c + make arc4cat + rm arc4cat.c +} + +usage() { + echo "Looks like you are doing it wrong. Try one of these:" + echo + echo "$0 analyze" + echo "$0 build" + echo "$0 cat" + echo "$0 collect" + return 1 +} + +main() { + local mode="${1}" + + shift + set -e + prepare + + case "${mode}" in + analyze) + analyze_collected_entropy "${@}" + ;; + build) + build_arc4cat + ;; + cat) + cat_collected_entropy + ;; + collect) + collect_entropy "${@}" + ;; + remix) + create_entropy_cache + ;; + *) + usage + ;; + esac +} + +main "${@}" -- 2.37.1 From 784e240b189cb4252ed61cc7488a43728f0f7b99 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 May 2015 19:06:37 +0200 Subject: [PATCH 092/310] release: Add ElectroBSD install instructions ... that aren't particularly verbose, though. Obtained from: ElectroBSD --- release/rc.local | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/release/rc.local b/release/rc.local index 8162cd1efff6..7acf5070ef5e 100755 --- a/release/rc.local +++ b/release/rc.local @@ -33,7 +33,7 @@ if [ $? -eq 0 ]; then else # Serial or other console echo - echo "Welcome to FreeBSD!" + echo "Welcome to ElectroBSD!" echo echo "Please choose the appropriate terminal type for your system." echo "Common console types are:" @@ -58,7 +58,19 @@ if [ -f /etc/installerconfig ]; then exit fi -dialog --backtitle "FreeBSD Installer" --title "Welcome" --extra-button --extra-label "Shell" --ok-label "Install" --cancel-label "Live CD" --yesno "Welcome to FreeBSD! Would you like to begin an installation or use the live CD?" 0 0 +local timeout=15 +if dialog --backtitle "ElectroBSD $(uname -m) on a stick in da house" \ + --title "Careful now, your data may be at risk." \ + --timeout ${timeout} \ + --yesno "To install ElectroBSD, execute cloudiatr after creating a cloudiatr.conf. Understood? You have ${timeout} seconds to respond." \ + 0 0; then + echo "Great. Good luck." +else + echo "That's very unfortunate. If you don't have backups you probably should not continue." +fi + +exit 0; + case $? in $DIALOG_OK) # Install -- 2.37.1 From 033446ce2574d52758031faa10c1225c1f4fe98c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 1 Aug 2015 11:42:55 +0200 Subject: [PATCH 093/310] sys/dev/vt: Default to showing splash CPU logo(s) on boot Obtained from: ElectroBSD --- sys/dev/vt/vt_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/vt/vt_core.c b/sys/dev/vt/vt_core.c index 85ab3d4cd73e..cbe27f4c8d1c 100644 --- a/sys/dev/vt/vt_core.c +++ b/sys/dev/vt/vt_core.c @@ -148,7 +148,7 @@ static VT_SYSCTL_INT(kbd_panic, 0, "Enable request to panic. " /* Used internally, not a tunable. */ int vt_draw_logo_cpus; -VT_SYSCTL_INT(splash_cpu, 0, "Show logo CPUs during boot"); +VT_SYSCTL_INT(splash_cpu, 1, "Show logo CPUs during boot"); VT_SYSCTL_INT(splash_ncpu, 0, "Override number of logos displayed " "(0 = do not override)"); VT_SYSCTL_INT(splash_cpu_style, 2, "Draw logo style " -- 2.37.1 From 61914e85172e6d73d6d1457ab2460a331336bb3a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 1 Aug 2015 16:59:55 +0200 Subject: [PATCH 094/310] sys/dev/vt: Default to showing Beastie instead of the orb/sextoy Obtained from: ElectroBSD --- sys/dev/vt/vt_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/vt/vt_core.c b/sys/dev/vt/vt_core.c index cbe27f4c8d1c..eac3c2f052a6 100644 --- a/sys/dev/vt/vt_core.c +++ b/sys/dev/vt/vt_core.c @@ -151,7 +151,7 @@ int vt_draw_logo_cpus; VT_SYSCTL_INT(splash_cpu, 1, "Show logo CPUs during boot"); VT_SYSCTL_INT(splash_ncpu, 0, "Override number of logos displayed " "(0 = do not override)"); -VT_SYSCTL_INT(splash_cpu_style, 2, "Draw logo style " +VT_SYSCTL_INT(splash_cpu_style, 1, "Draw logo style " "(0 = Alternate beastie, 1 = Beastie, 2 = Orb)"); VT_SYSCTL_INT(splash_cpu_duration, 10, "Hide logos after (seconds)"); -- 2.37.1 From 8a1e1b9e55bd1a24d15947036f1da5eb5dce190f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 2 Aug 2015 19:41:56 +0200 Subject: [PATCH 095/310] crypto/openssl: Use "ElectroBSD" in OPENSSL_VERSION_TEXT Obtained from: ElectroBSD --- crypto/openssl/include/openssl/opensslv.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crypto/openssl/include/openssl/opensslv.h b/crypto/openssl/include/openssl/opensslv.h index eb3e14013979..7d91b600690f 100644 --- a/crypto/openssl/include/openssl/opensslv.h +++ b/crypto/openssl/include/openssl/opensslv.h @@ -40,7 +40,7 @@ extern "C" { * major minor fix final patch/beta) */ # define OPENSSL_VERSION_NUMBER 0x1010111fL -# define OPENSSL_VERSION_TEXT "OpenSSL 1.1.1q-freebsd 5 Jul 2022" +# define OPENSSL_VERSION_TEXT "OpenSSL 1.1.1q-ElectroBSD 2022-07-05" /*- * The macros below are to be used for shared library (.so, .dll, ...) -- 2.37.1 From ba939158480a69a179708553f8e8eb36bd9ee5a6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 15 Aug 2015 11:29:39 +0200 Subject: [PATCH 096/310] ttys: Reduce default number of active terminals and mark the console as insecure Obtained from: ElectroBSD --- sbin/init/ttys | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/sbin/init/ttys b/sbin/init/ttys index a2f5b8975ecd..31dd65de3c71 100644 --- a/sbin/init/ttys +++ b/sbin/init/ttys @@ -27,17 +27,17 @@ # # If console is marked "insecure", then init will ask for the root password # when going to single-user mode. -console none unknown off secure +console none unknown off insecure # ttyv0 "/usr/libexec/getty Pc" xterm onifexists secure # Virtual terminals -ttyv1 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv2 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv3 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv4 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv5 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv6 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv7 "/usr/libexec/getty Pc" xterm onifexists secure +ttyv1 "/usr/libexec/getty Pc" xterm off secure +ttyv2 "/usr/libexec/getty Pc" xterm off secure +ttyv3 "/usr/libexec/getty Pc" xterm off secure +ttyv4 "/usr/libexec/getty Pc" xterm off secure +ttyv5 "/usr/libexec/getty Pc" xterm off secure +ttyv6 "/usr/libexec/getty Pc" xterm off secure +ttyv7 "/usr/libexec/getty Pc" xterm off secure ttyv8 "/usr/local/bin/xdm -nodaemon" xterm off secure # Serial terminals # The 'dialup' keyword identifies dialin lines to login, fingerd etc. -- 2.37.1 From ab4b83f87016f7d562c511dbc078ff81e8883e5f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 21:28:15 +0200 Subject: [PATCH 097/310] usr.sbin: Don't build uathload if MK_SOURCELESS_UCODE is defined It breaks the build if the source tree does not contain the firmware. Obtained from: ElectroBSD --- usr.sbin/Makefile | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index a961d0626568..af685e75c11c 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -200,7 +200,12 @@ SUBDIR.${MK_TCP_WRAPPERS}+= tcpdmatch SUBDIR.${MK_TOOLCHAIN}+= config SUBDIR.${MK_TOOLCHAIN}+= crunch SUBDIR.${MK_UNBOUND}+= unbound +# MK_SOURCELESS_UCODE is supposed to affect kernel +# modules only but there is no matching define for +# userland stuff. +.if ${MK_SOURCELESS_UCODE} != "no" SUBDIR.${MK_USB}+= uathload +.endif SUBDIR.${MK_USB}+= uhsoctl SUBDIR.${MK_USB}+= usbconfig SUBDIR.${MK_USB}+= usbdump -- 2.37.1 From e836ae6cc7361827100810f4a5fec28f3972ce3b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 21:33:19 +0200 Subject: [PATCH 098/310] Detach fwcontrol from the build It's pointless without firewire(4). Obtained from: ElectroBSD --- usr.sbin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index af685e75c11c..43b258b0f6a0 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -26,7 +26,6 @@ SUBDIR= adduser \ extattrctl \ fifolog \ fstyp \ - fwcontrol \ getfmac \ getpmac \ gstat \ -- 2.37.1 From b911fe9403117c42e01ad6c8df73079e9a3e6f95 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 23:58:46 +0200 Subject: [PATCH 099/310] sys/x86: Remove another FreeBSD reference from log messages Obtained from: ElectroBSD --- sys/x86/x86/mp_x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/x86/x86/mp_x86.c b/sys/x86/x86/mp_x86.c index 289885fa6213..f328986ab21b 100644 --- a/sys/x86/x86/mp_x86.c +++ b/sys/x86/x86/mp_x86.c @@ -735,7 +735,7 @@ cpu_mp_announce(void) const char *hyperthread; struct topo_analysis topology; - printf("FreeBSD/SMP: "); + printf("ElectroBSD/SMP: "); if (topo_analyze(&topo_root, 1, &topology)) { printf("%d package(s)", topology.entities[TOPO_LEVEL_PKG]); if (topology.entities[TOPO_LEVEL_GROUP] > 1) @@ -756,7 +756,7 @@ cpu_mp_announce(void) printf("\n"); if (disabled_cpus) { - printf("FreeBSD/SMP Online: "); + printf("ElectroBSD/SMP Online: "); if (topo_analyze(&topo_root, 0, &topology)) { printf("%d package(s)", topology.entities[TOPO_LEVEL_PKG]); -- 2.37.1 From 9d1585fb5e6231dbabd4f202e944ef07153f9749 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 4 Jun 2016 11:38:43 +0200 Subject: [PATCH 100/310] Detach a couple of rc scripts that aren't relevant for ElectroBSD XXX: Among other things this detaches bgfsck which could still be considered relevant for the memstick images. Obtained from: ElectroBSD --- libexec/rc/rc.d/Makefile | 6 ------ 1 file changed, 6 deletions(-) diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index cb0e10a9b565..6819d95a5795 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -14,7 +14,6 @@ CONFS= DAEMON \ addswap \ adjkerntz \ archdep \ - bgfsck \ ${_blacklistd} \ bridge \ cfumass \ @@ -30,7 +29,6 @@ CONFS= DAEMON \ dhclient \ dmesg \ dumpon \ - fsck \ gbde \ geli \ geli2 \ @@ -62,7 +60,6 @@ CONFS= DAEMON \ mdconfig2 \ mountd \ msgs \ - natd \ netif \ netoptions \ netwait \ @@ -80,8 +77,6 @@ CONFS= DAEMON \ pflog \ pfsync \ powerd \ - ppp \ - pppoed \ pwcheck \ quota \ random \ @@ -98,7 +93,6 @@ CONFS= DAEMON \ savecore \ securelevel \ serial \ - sppp \ soft-protection \ statd \ static_arp \ -- 2.37.1 From cba72be04dc76ac1360e4f81a522dbe890fd971a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 9 Jul 2015 14:55:50 +0200 Subject: [PATCH 101/310] etc/pkg: Add and install ElectroBSD.conf and a pubkey file ... for testing purposes. While at it, detach FreeBSD.conf and the corresponding fingerprint file from the build. Obtained from: ElectroBSD --- share/keys/pkg/trusted/Makefile | 2 +- .../keys/pkg/trusted/electrobsd-test-key.conf | 2 ++ usr.sbin/pkg/ElectroBSD.conf | 21 +++++++++++++++++++ usr.sbin/pkg/Makefile | 7 ++++++- usr.sbin/pkg/electrobsd-test-repo.pub | 14 +++++++++++++ 5 files changed, 44 insertions(+), 2 deletions(-) create mode 100644 share/keys/pkg/trusted/electrobsd-test-key.conf create mode 100644 usr.sbin/pkg/ElectroBSD.conf create mode 100644 usr.sbin/pkg/electrobsd-test-repo.pub diff --git a/share/keys/pkg/trusted/Makefile b/share/keys/pkg/trusted/Makefile index a2f9d83cc59a..93c287f611a7 100644 --- a/share/keys/pkg/trusted/Makefile +++ b/share/keys/pkg/trusted/Makefile @@ -2,7 +2,7 @@ PACKAGE= pkg-bootstrap -FILES= pkg.freebsd.org.2013102301 +FILES= electrobsd-test-key.conf FILESDIR= ${SHAREDIR}/keys/pkg/trusted FILESMODE= 644 diff --git a/share/keys/pkg/trusted/electrobsd-test-key.conf b/share/keys/pkg/trusted/electrobsd-test-key.conf new file mode 100644 index 000000000000..e81a919e0ab2 --- /dev/null +++ b/share/keys/pkg/trusted/electrobsd-test-key.conf @@ -0,0 +1,2 @@ +function: "sha256" +fingerprint: "8b1ef90f16ccc99342fb204f7ff57cdd31449e5ce1baa10c0b1b09593c44219d" diff --git a/usr.sbin/pkg/ElectroBSD.conf b/usr.sbin/pkg/ElectroBSD.conf new file mode 100644 index 000000000000..4bf82d992bee --- /dev/null +++ b/usr.sbin/pkg/ElectroBSD.conf @@ -0,0 +1,21 @@ +ElectroBSD: { + # Note that accessing this repository requires a local proxy + # that port forwards the incomming http requests to an authorized + # Tor client that forwards them to: http://gkpssb7hd77qznoa.onion/ + # + # To get the required secret you have to ask someone who knows it and + # come up with a good reason why you want to run unreproducible packages + # instead of building your packages from source. + # + # Example bootstrapping setup: + # Your server: pkg -> sshd + # | + # Your client: --> ssh -> privoxy -> tor + # | + # Outside your control: --> Tor network -> Tor HS + url: "http://127.0.0.1:8000/packages/${ABI}/", + mirror_type: "NONE", + signature_type: "pubkey", + pubkey: "/etc/pkg/electrobsd-test-repo.pub", + enabled: yes +} diff --git a/usr.sbin/pkg/Makefile b/usr.sbin/pkg/Makefile index e40265146657..5cbab9090136 100644 --- a/usr.sbin/pkg/Makefile +++ b/usr.sbin/pkg/Makefile @@ -17,11 +17,16 @@ PKGCONFBRANCH?= latest . endif . endif .endif -PKGCONF?= FreeBSD.conf.${PKGCONFBRANCH} +PKGCONF?= ElectroBSD.conf CONFS= ${PKGCONF} CONFSNAME_${PKGCONF}= ${PKGCONF:C/\.conf.+$/.conf/} CONFSDIR= /etc/pkg CONFSMODE= 644 + +FILES= electrobsd-test-repo.pub +FILESDIR= /etc/pkg +FILESMODE= 644 + PROG= pkg SRCS= pkg.c dns_utils.c config.c MAN= pkg.7 diff --git a/usr.sbin/pkg/electrobsd-test-repo.pub b/usr.sbin/pkg/electrobsd-test-repo.pub new file mode 100644 index 000000000000..ee213cb062b1 --- /dev/null +++ b/usr.sbin/pkg/electrobsd-test-repo.pub @@ -0,0 +1,14 @@ +-----BEGIN PUBLIC KEY----- +MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAtwIgcSgQT2QJvxwDiKhh +xYidaxWaJL/EHXnYH8TpiujNeg8yZUCiT07MChw/wO3KrVY3SlhFkl8MHnwaFWNS +jSJFNQIHoVo8of0juDXhIMMY0DTVFVpSfgq+Ea0SrVZawQchcc7XODjp7mQOH0Nz +Z0FUpQ4z7TsdrPV/Kj/SQteK7Q0qAnDGT5GBxb+4m5qmNbv74GVpWmxJ5kC4OY3g +v8oTRZ5wG/mf5nKG+QXGKbcfoK+fd+NL9gp5awhvnp5cRo2+ZrQvnYKy5ym9O+jM +9hv5bL7OYIW7gErPdQd0+SFejcDg4TamY824krkjgIf7pnRVAuuSHiy8PaHmmNMd +StFo8IW6687C0X4gNuaPtR1TCkVZugrgev/w5GPp765HmE4WvrtPU4u9/EvWO2L6 +2bHmsJqNIGyq1lfHk/cx49CEhDPjRcYin8MKKzRFUR7CWiNDBz2GK6Lcfd78cG5I +HGJAthhi3rOOnLsTMuIQ/+GkVCDNwweLxSoLQUzGnheHqBjYyZSfZ8u9I/OKtwp3 +OMhGOD1/yuRaat+QER5VwiDP3AxcIlq6aNNytJW6l/kZZLVsr99YmROyYsTps0Zm +OXqxQFsZ6zzlYO95HP4ITJ7ubk84YPTvxbQ6LqsijqRI322aNvxxeEWyFxXpyCpp +uq50HtVoVX1a4ONsh/9l75sCAwEAAQ== +-----END PUBLIC KEY----- -- 2.37.1 From 87dce2df12b0fed7cd178d73e35d2f28c68c017c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 17 Sep 2015 17:20:31 +0200 Subject: [PATCH 102/310] share/mk/src.opts.mk: Detach LIB32 from the build Obtained from: ElectroBSD --- share/mk/src.opts.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 992a397735a1..16d6b62c74b8 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -244,6 +244,7 @@ __DEFAULT_NO_OPTIONS += \ GCOV \ IPFILTER \ ISCSI \ + LIB32 \ NDIS \ RBOOTD \ PC_SYSINSTALL \ -- 2.37.1 From 01dfea741e4e67ee14adc87f2019238fe74e1a66 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 12:53:54 +0200 Subject: [PATCH 103/310] fortune/datafiles: Add a bunch of German fundamental rights Currently they are somewhat poorly formatted and the list only includes those rights that are defined in the German Grundgesetz itself, not those rights that were derived by the Bundesverfassungsgericht. Enforcing these rights is left as an excercise for the reader ... Obtained from: ElectroBSD --- usr.bin/fortune/datfiles/Makefile | 2 +- usr.bin/fortune/datfiles/grundrechte | 346 +++++++++++++++++++++++++++ 2 files changed, 347 insertions(+), 1 deletion(-) create mode 100644 usr.bin/fortune/datfiles/grundrechte diff --git a/usr.bin/fortune/datfiles/Makefile b/usr.bin/fortune/datfiles/Makefile index 80b8cd2ab641..ea87b1083a91 100644 --- a/usr.bin/fortune/datfiles/Makefile +++ b/usr.bin/fortune/datfiles/Makefile @@ -1,7 +1,7 @@ # @(#)Makefile 8.2 (Berkeley) 4/19/94 # $FreeBSD$ -DB= freebsd-tips +DB= grundrechte freebsd-tips BLDS= ${DB:S/$/.dat/} FILES= ${DB} ${BLDS} diff --git a/usr.bin/fortune/datfiles/grundrechte b/usr.bin/fortune/datfiles/grundrechte new file mode 100644 index 000000000000..4eedc46ccfc8 --- /dev/null +++ b/usr.bin/fortune/datfiles/grundrechte @@ -0,0 +1,346 @@ +% +Grundgesetz Artikel 1 +(1) Die Würde des Menschen ist unantastbar. Sie zu achten und zu +schützen ist Verpflichtung aller staatlichen Gewalt. +(2) Das Deutsche Volk bekennt sich darum zu unverletzlichen und +unveräußerlichen Menschenrechten als Grundlage jeder menschlichen +Gemeinschaft, des Friedens und der Gerechtigkeit in der Welt. +(3) Die nachfolgenden Grundrechte binden Gesetzgebung, vollziehende +Gewalt und Rechtsprechung als unmittelbar geltendes Recht. +% +Grundgesetz Artikel 2 +(1) Jeder hat das Recht auf die freie Entfaltung seiner +Persönlichkeit, soweit er nicht die Rechte anderer verletzt und nicht +gegen die verfassungsmäßige Ordnung oder das Sittengesetz verstößt. +(2) Jeder hat das Recht auf Leben und körperliche Unversehrtheit. Die +Freiheit der Person ist unverletzlich. In diese Rechte darf nur auf +Grund eines Gesetzes eingegriffen werden. +% +Grundgesetz Artikel 3 +(1) Alle Menschen sind vor dem Gesetz gleich. +(2) Männer und Frauen sind gleichberechtigt. Der Staat fördert die +tatsächliche Durchsetzung der Gleichberechtigung von Frauen und +Männern und wirkt auf die Beseitigung bestehender Nachteile hin. +(3) Niemand darf wegen seines Geschlechtes, seiner Abstammung, seiner +Rasse, seiner Sprache, seiner Heimat und Herkunft, seines Glaubens, +seiner religiösen oder politischen Anschauungen benachteiligt oder +bevorzugt werden. Niemand darf wegen seiner Behinderung benachteiligt +werden. +% +Grundgesetz Artikel 4 +(1) Die Freiheit des Glaubens, des Gewissens und die Freiheit des +religiösen und weltanschaulichen Bekenntnisses sind unverletzlich. +(2) Die ungestörte Religionsausübung wird gewährleistet. +(3) Niemand darf gegen sein Gewissen zum Kriegsdienst mit der Waffe +gezwungen werden. Das Nähere regelt ein Bundesgesetz. +% +Grundgesetz Artikel 5 +(1) Jeder hat das Recht, seine Meinung in Wort, Schrift und Bild frei +zu äußern und zu verbreiten und sich aus allgemein zugänglichen +Quellen ungehindert zu unterrichten. Die Pressefreiheit und die +Freiheit der Berichterstattung durch Rundfunk und Film werden +gewährleistet. Eine Zensur findet nicht statt. +(2) Diese Rechte finden ihre Schranken in den Vorschriften der +allgemeinen Gesetze, den gesetzlichen Bestimmungen zum Schutze der +Jugend und in dem Recht der persönlichen Ehre. +(3) Kunst und Wissenschaft, Forschung und Lehre sind frei. Die +Freiheit der Lehre entbindet nicht von der Treue zur Verfassung. +% +Grundgesetz Artikel 6 +(1) Ehe und Familie stehen unter dem besonderen Schutze der staatlichen Ordnung. +(2) Pflege und Erziehung der Kinder sind das natürliche Recht der +Eltern und die zuvörderst ihnen obliegende Pflicht. Über ihre +Betätigung wacht die staatliche Gemeinschaft. +(3) Gegen den Willen der Erziehungsberechtigten dürfen Kinder nur auf +Grund eines Gesetzes von der Familie getrennt werden, wenn die +Erziehungsberechtigten versagen oder wenn die Kinder aus anderen +Gründen zu verwahrlosen drohen. +(4) Jede Mutter hat Anspruch auf den Schutz und die Fürsorge der Gemeinschaft. +(5) Den unehelichen Kindern sind durch die Gesetzgebung die gleichen +Bedingungen für ihre leibliche und seelische Entwicklung und ihre +Stellung in der Gesellschaft zu schaffen wie den ehelichen Kindern. +% +Grundgesetz Artikel 7 +(1) Das gesamte Schulwesen steht unter der Aufsicht des Staates. +(2) Die Erziehungsberechtigten haben das Recht, über die Teilnahme des +Kindes am Religionsunterricht zu bestimmen. +(3) Der Religionsunterricht ist in den öffentlichen Schulen mit +Ausnahme der bekenntnisfreien Schulen ordentliches +Lehrfach. Unbeschadet des staatlichen Aufsichtsrechtes wird der +Religionsunterricht in Übereinstimmung mit den Grundsätzen der +Religionsgemeinschaften erteilt. Kein Lehrer darf gegen seinen Willen +verpflichtet werden, Religionsunterricht zu erteilen. +(4) Das Recht zur Errichtung von privaten Schulen wird +gewährleistet. Private Schulen als Ersatz für öffentliche Schulen +bedürfen der Genehmigung des Staates und unterstehen den +Landesgesetzen. Die Genehmigung ist zu erteilen, wenn die privaten +Schulen in ihren Lehrzielen und Einrichtungen sowie in der +wissenschaftlichen Ausbildung ihrer Lehrkräfte nicht hinter den +öffentlichen Schulen zurückstehen und eine Sonderung der Schüler nach +den Besitzverhältnissen der Eltern nicht gefördert wird. Die +Genehmigung ist zu versagen, wenn die wirtschaftliche und rechtliche +Stellung der Lehrkräfte nicht genügend gesichert ist. +(5) Eine private Volksschule ist nur zuzulassen, wenn die +Unterrichtsverwaltung ein besonderes pädagogisches Interesse anerkennt +oder, auf Antrag von Erziehungsberechtigten, wenn sie als +Gemeinschaftsschule, als Bekenntnis- oder Weltanschauungsschule +errichtet werden soll und eine öffentliche Volksschule dieser Art in +der Gemeinde nicht besteht. +(6) Vorschulen bleiben aufgehoben. +% +Grundgesetz Artikel 8 +(1) Alle Deutschen haben das Recht, sich ohne Anmeldung oder Erlaubnis +friedlich und ohne Waffen zu versammeln. +(2) Für Versammlungen unter freiem Himmel kann dieses Recht durch +Gesetz oder auf Grund eines Gesetzes beschränkt werden. +% +Grundgesetz Artikel 9 +(1) Alle Deutschen haben das Recht, Vereine und Gesellschaften zu bilden. +(2) Vereinigungen, deren Zwecke oder deren Tätigkeit den Strafgesetzen +zuwiderlaufen oder die sich gegen die verfassungsmäßige Ordnung oder +gegen den Gedanken der Völkerverständigung richten, sind verboten. +(3) Das Recht, zur Wahrung und Förderung der Arbeits- und +Wirtschaftsbedingungen Vereinigungen zu bilden, ist für jedermann und +für alle Berufe gewährleistet. Abreden, die dieses Recht einschränken +oder zu behindern suchen, sind nichtig, hierauf gerichtete Maßnahmen +sind rechtswidrig. Maßnahmen nach den Artikeln 12a, 35 Abs. 2 und 3, +Artikel 87a Abs. 4 und Artikel 91 dürfen sich nicht gegen +Arbeitskämpfe richten, die zur Wahrung und Förderung der Arbeits- und +Wirtschaftsbedingungen von Vereinigungen im Sinne des Satzes 1 geführt +werden. +% +Grundgesetz Artikel 10 +(1) Das Briefgeheimnis sowie das Post- und Fernmeldegeheimnis sind +unverletzlich. +(2) Beschränkungen dürfen nur auf Grund eines Gesetzes angeordnet +werden. Dient die Beschränkung dem Schutze der freiheitlichen +demokratischen Grundordnung oder des Bestandes oder der Sicherung des +Bundes oder eines Landes, so kann das Gesetz bestimmen, daß sie dem +Betroffenen nicht mitgeteilt wird und daß an die Stelle des +Rechtsweges die Nachprüfung durch von der Volksvertretung bestellte +Organe und Hilfsorgane tritt. +% +Grundgesetz Artikel 11 +(1) Alle Deutschen genießen Freizügigkeit im ganzen Bundesgebiet. +(2) Dieses Recht darf nur durch Gesetz oder auf Grund eines Gesetzes +und nur für die Fälle eingeschränkt werden, in denen eine ausreichende +Lebensgrundlage nicht vorhanden ist und der Allgemeinheit daraus +besondere Lasten entstehen würden oder in denen es zur Abwehr einer +drohenden Gefahr für den Bestand oder die freiheitliche demokratische +Grundordnung des Bundes oder eines Landes, zur Bekämpfung von +Seuchengefahr, Naturkatastrophen oder besonders schweren +Unglücksfällen, zum Schutze der Jugend vor Verwahrlosung oder um +strafbaren Handlungen vorzubeugen, erforderlich ist. +% +Grundgesetz Artikel 12 +(1) Alle Deutschen haben das Recht, Beruf, Arbeitsplatz und +Ausbildungsstätte frei zu wählen. Die Berufsausübung kann durch Gesetz +oder auf Grund eines Gesetzes geregelt werden. +(2) Niemand darf zu einer bestimmten Arbeit gezwungen werden, außer im +Rahmen einer herkömmlichen allgemeinen, für alle gleichen öffentlichen +Dienstleistungspflicht. +(3) Zwangsarbeit ist nur bei einer gerichtlich angeordneten +Freiheitsentziehung zulässig. +% +Grundgesetz Artikel 12a +(1) Männer können vom vollendeten achtzehnten Lebensjahr an zum Dienst +in den Streitkräften, im Bundesgrenzschutz oder in einem +Zivilschutzverband verpflichtet werden. +(2) Wer aus Gewissensgründen den Kriegsdienst mit der Waffe +verweigert, kann zu einem Ersatzdienst verpflichtet werden. Die Dauer +des Ersatzdienstes darf die Dauer des Wehrdienstes nicht +übersteigen. Das Nähere regelt ein Gesetz, das die Freiheit der +Gewissensentscheidung nicht beeinträchtigen darf und auch eine +Möglichkeit des Ersatzdienstes vorsehen muß, die in keinem +Zusammenhang mit den Verbänden der Streitkräfte und des +Bundesgrenzschutzes steht. +(3) Wehrpflichtige, die nicht zu einem Dienst nach Absatz 1 oder 2 +herangezogen sind, können im Verteidigungsfalle durch Gesetz oder auf +Grund eines Gesetzes zu zivilen Dienstleistungen für Zwecke der +Verteidigung einschließlich des Schutzes der Zivilbevölkerung in +Arbeitsverhältnisse verpflichtet werden; Verpflichtungen in +öffentlich-rechtliche Dienstverhältnisse sind nur zur Wahrnehmung +polizeilicher Aufgaben oder solcher hoheitlichen Aufgaben der +öffentlichen Verwaltung, die nur in einem öffentlich-rechtlichen +Dienstverhältnis erfüllt werden können, zulässig. Arbeitsverhältnisse +nach Satz 1 können bei den Streitkräften, im Bereich ihrer Versorgung +sowie bei der öffentlichen Verwaltung begründet werden; +Verpflichtungen in Arbeitsverhältnisse im Bereiche der Versorgung der +Zivilbevölkerung sind nur zulässig, um ihren lebensnotwendigen Bedarf +zu decken oder ihren Schutz sicherzustellen. +(4) Kann im Verteidigungsfalle der Bedarf an zivilen Dienstleistungen +im zivilen Sanitäts- und Heilwesen sowie in der ortsfesten +militärischen Lazarettorganisation nicht auf freiwilliger Grundlage +gedeckt werden, so können Frauen vom vollendeten achtzehnten bis zum +vollendeten fünfundfünfzigsten Lebensjahr durch Gesetz oder auf Grund +eines Gesetzes zu derartigen Dienstleistungen herangezogen werden. Sie +dürfen auf keinen Fall zum Dienst mit der Waffe verpflichtet werden. +(5) Für die Zeit vor dem Verteidigungsfalle können Verpflichtungen +nach Absatz 3 nur nach Maßgabe des Artikels 80a Abs. 1 begründet +werden. Zur Vorbereitung auf Dienstleistungen nach Absatz 3, für die +besondere Kenntnisse oder Fertigkeiten erforderlich sind, kann durch +Gesetz oder auf Grund eines Gesetzes die Teilnahme an +Ausbildungsveranstaltungen zur Pflicht gemacht werden. Satz 1 findet +insoweit keine Anwendung. +(6) Kann im Verteidigungsfalle der Bedarf an Arbeitskräften für die in +Absatz 3 Satz 2 genannten Bereiche auf freiwilliger Grundlage nicht +gedeckt werden, so kann zur Sicherung dieses Bedarfs die Freiheit der +Deutschen, die Ausübung eines Berufs oder den Arbeitsplatz aufzugeben, +durch Gesetz oder auf Grund eines Gesetzes eingeschränkt werden. Vor +Eintritt des Verteidigungsfalles gilt Absatz 5 Satz 1 entsprechend. +% +Grundgesetz Artikel 13 +(1) Die Wohnung ist unverletzlich. +(2) Durchsuchungen dürfen nur durch den Richter, bei Gefahr im Verzuge +auch durch die in den Gesetzen vorgesehenen anderen Organe angeordnet +und nur in der dort vorgeschriebenen Form durchgeführt werden. +(3) Begründen bestimmte Tatsachen den Verdacht, daß jemand eine durch +Gesetz einzeln bestimmte besonders schwere Straftat begangen hat, so +dürfen zur Verfolgung der Tat auf Grund richterlicher Anordnung +technische Mittel zur akustischen Überwachung von Wohnungen, in denen +der Beschuldigte sich vermutlich aufhält, eingesetzt werden, wenn die +Erforschung des Sachverhalts auf andere Weise unverhältnismäßig +erschwert oder aussichtslos wäre. Die Maßnahme ist zu befristen. Die +Anordnung erfolgt durch einen mit drei Richtern besetzten +Spruchkörper. Bei Gefahr im Verzuge kann sie auch durch einen +einzelnen Richter getroffen werden. +(4) Zur Abwehr dringender Gefahren für die öffentliche Sicherheit, +insbesondere einer gemeinen Gefahr oder einer Lebensgefahr, dürfen +technische Mittel zur Überwachung von Wohnungen nur auf Grund +richterlicher Anordnung eingesetzt werden. Bei Gefahr im Verzuge kann +die Maßnahme auch durch eine andere gesetzlich bestimmte Stelle +angeordnet werden; eine richterliche Entscheidung ist unverzüglich +nachzuholen. +(5) Sind technische Mittel ausschließlich zum Schutze der bei einem +Einsatz in Wohnungen tätigen Personen vorgesehen, kann die Maßnahme +durch eine gesetzlich bestimmte Stelle angeordnet werden. Eine +anderweitige Verwertung der hierbei erlangten Erkenntnisse ist nur zum +Zwecke der Strafverfolgung oder der Gefahrenabwehr und nur zulässig, +wenn zuvor die Rechtmäßigkeit der Maßnahme richterlich festgestellt +ist; bei Gefahr im Verzuge ist die richterliche Entscheidung +unverzüglich nachzuholen. +(6) Die Bundesregierung unterrichtet den Bundestag jährlich über den +nach Absatz 3 sowie über den im Zuständigkeitsbereich des Bundes nach +Absatz 4 und, soweit richterlich überprüfungsbedürftig, nach Absatz 5 +erfolgten Einsatz technischer Mittel. Ein vom Bundestag gewähltes +Gremium übt auf der Grundlage dieses Berichts die parlamentarische +Kontrolle aus. Die Länder gewährleisten eine gleichwertige +parlamentarische Kontrolle. +(7) Eingriffe und Beschränkungen dürfen im übrigen nur zur Abwehr +einer gemeinen Gefahr oder einer Lebensgefahr für einzelne Personen, +auf Grund eines Gesetzes auch zur Verhütung dringender Gefahren für +die öffentliche Sicherheit und Ordnung, insbesondere zur Behebung der +Raumnot, zur Bekämpfung von Seuchengefahr oder zum Schutze gefährdeter +Jugendlicher vorgenommen werden. +% +Grundgesetz Artikel 14 +(1) Das Eigentum und das Erbrecht werden gewährleistet. Inhalt und +Schranken werden durch die Gesetze bestimmt. +(2) Eigentum verpflichtet. Sein Gebrauch soll zugleich dem Wohle der +Allgemeinheit dienen. +(3) Eine Enteignung ist nur zum Wohle der Allgemeinheit zulässig. Sie +darf nur durch Gesetz oder auf Grund eines Gesetzes erfolgen, das Art +und Ausmaß der Entschädigung regelt. Die Entschädigung ist unter +gerechter Abwägung der Interessen der Allgemeinheit und der +Beteiligten zu bestimmen. Wegen der Höhe der Entschädigung steht im +Streitfalle der Rechtsweg vor den ordentlichen Gerichten offen. +% +Grundgesetz Artikel 15 +Grund und Boden, Naturschätze und Produktionsmittel können zum Zwecke +der Vergesellschaftung durch ein Gesetz, das Art und Ausmaß der +Entschädigung regelt, in Gemeineigentum oder in andere Formen der +Gemeinwirtschaft überführt werden. Für die Entschädigung gilt Artikel +14 Abs. 3 Satz 3 und 4 entsprechend. +% +Grundgesetz Artikel 16 +(1) Die deutsche Staatsangehörigkeit darf nicht entzogen werden. Der +Verlust der Staatsangehörigkeit darf nur auf Grund eines Gesetzes und +gegen den Willen des Betroffenen nur dann eintreten, wenn der +Betroffene dadurch nicht staatenlos wird. +(2) Kein Deutscher darf an das Ausland ausgeliefert werden. Durch +Gesetz kann eine abweichende Regelung für Auslieferungen an einen +Mitgliedstaat der Europäischen Union oder an einen internationalen +Gerichtshof getroffen werden, soweit rechtsstaatliche Grundsätze +gewahrt sind. +% +Grundgesetz Artikel 16a +(1) Politisch Verfolgte genießen Asylrecht. +(2) Auf Absatz 1 kann sich nicht berufen, wer aus einem Mitgliedstaat +der Europäischen Gemeinschaften oder aus einem anderen Drittstaat +einreist, in dem die Anwendung des Abkommens über die Rechtsstellung +der Flüchtlinge und der Konvention zum Schutze der Menschenrechte und +Grundfreiheiten sichergestellt ist. Die Staaten außerhalb der +Europäischen Gemeinschaften, auf die die Voraussetzungen des Satzes 1 +zutreffen, werden durch Gesetz, das der Zustimmung des Bundesrates +bedarf, bestimmt. In den Fällen des Satzes 1 können +aufenthaltsbeendende Maßnahmen unabhängig von einem hiergegen +eingelegten Rechtsbehelf vollzogen werden. +(3) Durch Gesetz, das der Zustimmung des Bundesrates bedarf, können +Staaten bestimmt werden, bei denen auf Grund der Rechtslage, der +Rechtsanwendung und der allgemeinen politischen Verhältnisse +gewährleistet erscheint, daß dort weder politische Verfolgung noch +unmenschliche oder erniedrigende Bestrafung oder Behandlung +stattfindet. Es wird vermutet, daß ein Ausländer aus einem solchen +Staat nicht verfolgt wird, solange er nicht Tatsachen vorträgt, die +die Annahme begründen, daß er entgegen dieser Vermutung politisch +verfolgt wird. +(4) Die Vollziehung aufenthaltsbeendender Maßnahmen wird in den Fällen +des Absatzes 3 und in anderen Fällen, die offensichtlich unbegründet +sind oder als offensichtlich unbegründet gelten, durch das Gericht nur +ausgesetzt, wenn ernstliche Zweifel an der Rechtmäßigkeit der Maßnahme +bestehen; der Prüfungsumfang kann eingeschränkt werden und verspätetes +Vorbringen unberücksichtigt bleiben. Das Nähere ist durch Gesetz zu +bestimmen. +(5) Die Absätze 1 bis 4 stehen völkerrechtlichen Verträgen von +Mitgliedstaaten der Europäischen Gemeinschaften untereinander und mit +dritten Staaten nicht entgegen, die unter Beachtung der +Verpflichtungen aus dem Abkommen über die Rechtsstellung der +Flüchtlinge und der Konvention zum Schutze der Menschenrechte und +Grundfreiheiten, deren Anwendung in den Vertragsstaaten sichergestellt +sein muß, Zuständigkeitsregelungen für die Prüfung von Asylbegehren +einschließlich der gegenseitigen Anerkennung von Asylentscheidungen +treffen. +% +Grundgesetz Artikel 17 +Jedermann hat das Recht, sich einzeln oder in Gemeinschaft mit anderen +schriftlich mit Bitten oder Beschwerden an die zuständigen Stellen und +an die Volksvertretung zu wenden. +% +Grundgesetz Artikel 17a +(1) Gesetze über Wehrdienst und Ersatzdienst können bestimmen, daß für +die Angehörigen der Streitkräfte und des Ersatzdienstes während der +Zeit des Wehr- oder Ersatzdienstes das Grundrecht, seine Meinung in +Wort, Schrift und Bild frei zu äußern und zu verbreiten (Artikel 5 +Abs. 1 Satz 1 erster Halbsatz), das Grundrecht der +Versammlungsfreiheit (Artikel 8) und das Petitionsrecht (Artikel 17), +soweit es das Recht gewährt, Bitten oder Beschwerden in Gemeinschaft +mit anderen vorzubringen, eingeschränkt werden. +(2) Gesetze, die der Verteidigung einschließlich des Schutzes der +Zivilbevölkerung dienen, können bestimmen, daß die Grundrechte der +Freizügigkeit (Artikel 11) und der Unverletzlichkeit der Wohnung +(Artikel 13) eingeschränkt werden. +% +Grundgesetz Artikel 18 +Wer die Freiheit der Meinungsäußerung, insbesondere die Pressefreiheit +(Artikel 5 Abs. 1), die Lehrfreiheit (Artikel 5 Abs. 3), die +Versammlungsfreiheit (Artikel 8), die Vereinigungsfreiheit (Artikel +9), das Brief-, Post- und Fernmeldegeheimnis (Artikel 10), das +Eigentum (Artikel 14) oder das Asylrecht (Artikel 16a) zum Kampfe +gegen die freiheitliche demokratische Grundordnung mißbraucht, +verwirkt diese Grundrechte. Die Verwirkung und ihr Ausmaß werden durch +das Bundesverfassungsgericht ausgesprochen. +% +Grundgesetz Artikel 19 +(1) Soweit nach diesem Grundgesetz ein Grundrecht durch Gesetz oder +auf Grund eines Gesetzes eingeschränkt werden kann, muß das Gesetz +allgemein und nicht nur für den Einzelfall gelten. Außerdem muß das +Gesetz das Grundrecht unter Angabe des Artikels nennen. +(2) In keinem Falle darf ein Grundrecht in seinem Wesensgehalt +angetastet werden. +(3) Die Grundrechte gelten auch für inländische juristische Personen, +soweit sie ihrem Wesen nach auf diese anwendbar sind. +(4) Wird jemand durch die öffentliche Gewalt in seinen Rechten +verletzt, so steht ihm der Rechtsweg offen. Soweit eine andere +Zuständigkeit nicht begründet ist, ist der ordentliche Rechtsweg +gegeben. Artikel 10 Abs. 2 Satz 2 bleibt unberührt. +% -- 2.37.1 From 3649a289db1505d2aafd908e0d83b155157ede37 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 13:16:50 +0200 Subject: [PATCH 104/310] share/skel/dot.profile: Change fortune file to "grundrechte" Obtained from: ElectroBSD --- share/skel/dot.profile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/skel/dot.profile b/share/skel/dot.profile index 02623293b592..dc3c0e9b8e60 100644 --- a/share/skel/dot.profile +++ b/share/skel/dot.profile @@ -27,4 +27,4 @@ if [ "$PWD" != "$HOME" ] && [ "$PWD" -ef "$HOME" ] ; then cd ; fi if [ -x /usr/bin/resizewin ] ; then /usr/bin/resizewin -z ; fi # Display a random cookie on each login. -if [ -x /usr/bin/fortune ] ; then /usr/bin/fortune freebsd-tips ; fi +if [ -x /usr/bin/fortune ] ; then /usr/bin/fortune grundrechte ; fi -- 2.37.1 From 10c494e353f62430e8eceea4ae8808e9486209d4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 18 Aug 2015 09:57:30 +0200 Subject: [PATCH 105/310] sys/x86: Note that ElectroBSD does not protect against compromised hosts ... if a hypervisor has been detected. Obtained from: ElectroBSD --- sys/x86/x86/identcpu.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/sys/x86/x86/identcpu.c b/sys/x86/x86/identcpu.c index 263e05dcf0ae..e1670ec23b91 100644 --- a/sys/x86/x86/identcpu.c +++ b/sys/x86/x86/identcpu.c @@ -2666,8 +2666,15 @@ static void print_hypervisor_info(void) { - if (*hv_vendor != '\0') + if (*hv_vendor != '\0') { printf("Hypervisor: Origin = \"%s\"\n", hv_vendor); + printf("ElectroBSD security advise: It looks like " + "you may be clown computing.\n" + "Please note that the people controlling the " + "'cloud' this system is\n" + "running on can monitor and sabotage everything " + "it is doing.\n"); + } } /* -- 2.37.1 From e2defa42bd6d2959ff276ea959fbc429d15812c7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 8 Jan 2017 23:11:47 +0100 Subject: [PATCH 106/310] Detach enigma(1) from the build Obtained from: ElectroBSD --- usr.bin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/usr.bin/Makefile b/usr.bin/Makefile index fc6b94adf4c2..62569bedb151 100644 --- a/usr.bin/Makefile +++ b/usr.bin/Makefile @@ -35,7 +35,6 @@ SUBDIR= alias \ du \ elfctl \ elfdump \ - enigma \ env \ etdump \ expand \ -- 2.37.1 From f712b55763bc1265560502473ca806b4106e36ae Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 9 Jan 2016 00:37:52 +0100 Subject: [PATCH 107/310] crontab: Default to not sending out (most-likely unencrypted) mails Obtained from: ElectroBSD --- usr.sbin/cron/cron/crontab | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/usr.sbin/cron/cron/crontab b/usr.sbin/cron/cron/crontab index 994d4ae6af00..cbcff76d64bf 100644 --- a/usr.sbin/cron/cron/crontab +++ b/usr.sbin/cron/cron/crontab @@ -1,9 +1,8 @@ -# /etc/crontab - root's crontab for FreeBSD -# -# $FreeBSD$ +# /etc/crontab - root's crontab for ElectroBSD # SHELL=/bin/sh PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin +MAILTO="" # #minute hour mday month wday who command # -- 2.37.1 From 9a894829b9211d8a070d9c9293faa795cf59bb59 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 16 Jan 2016 15:24:51 +0100 Subject: [PATCH 108/310] Detach unreproducible uzip tests (added in r293821) that aren't relevant for ElectroBSD Obtained from: ElectroBSD --- tests/sys/geom/class/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/sys/geom/class/Makefile b/tests/sys/geom/class/Makefile index 6e573d6164b6..beae0fd3d675 100644 --- a/tests/sys/geom/class/Makefile +++ b/tests/sys/geom/class/Makefile @@ -18,7 +18,8 @@ TESTS_SUBDIRS+= part TESTS_SUBDIRS+= raid3 TESTS_SUBDIRS+= shsec TESTS_SUBDIRS+= stripe -TESTS_SUBDIRS+= uzip +# Currently not reproducible +#TESTS_SUBDIRS+= uzip ${PACKAGE}FILES+= geom_subr.sh -- 2.37.1 From a9ba975c6c6109a1c37313efc3ff226bbccafc57 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Mar 2016 14:53:42 +0100 Subject: [PATCH 109/310] sshd: Disable UsePam, X11Forwarding and UseDNS by default (XXX disable UsePam in config) Obtained from: ElectroBSD --- crypto/openssh/servconf.c | 6 +++--- crypto/openssh/sshd_config | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/crypto/openssh/servconf.c b/crypto/openssh/servconf.c index 0c8e6f950e86..0a5c13bfd7c8 100644 --- a/crypto/openssh/servconf.c +++ b/crypto/openssh/servconf.c @@ -284,7 +284,7 @@ fill_default_server_options(ServerOptions *options) /* Portable-specific options */ if (options->use_pam == -1) - options->use_pam = 1; + options->use_pam = 0; /* Standard Options */ if (options->num_host_key_files == 0) { @@ -328,7 +328,7 @@ fill_default_server_options(ServerOptions *options) if (options->print_lastlog == -1) options->print_lastlog = 1; if (options->x11_forwarding == -1) - options->x11_forwarding = 1; + options->x11_forwarding = 0; if (options->x11_display_offset == -1) options->x11_display_offset = 10; if (options->x11_use_localhost == -1) @@ -415,7 +415,7 @@ fill_default_server_options(ServerOptions *options) if (options->max_sessions == -1) options->max_sessions = DEFAULT_SESSIONS_MAX; if (options->use_dns == -1) - options->use_dns = 1; + options->use_dns = 0; if (options->client_alive_interval == -1) options->client_alive_interval = 0; if (options->client_alive_count_max == -1) diff --git a/crypto/openssh/sshd_config b/crypto/openssh/sshd_config index 2ef36f63ae62..ebd1d9de81bc 100644 --- a/crypto/openssh/sshd_config +++ b/crypto/openssh/sshd_config @@ -75,7 +75,7 @@ AuthorizedKeysFile .ssh/authorized_keys #GSSAPIAuthentication no #GSSAPICleanupCredentials yes -# Set this to 'no' to disable PAM authentication, account processing, +# Set this to 'yes' to enable PAM authentication, account processing, # and session processing. If this is enabled, PAM authentication will # be allowed through the KbdInteractiveAuthentication and # PasswordAuthentication. Depending on your PAM configuration, @@ -89,7 +89,7 @@ AuthorizedKeysFile .ssh/authorized_keys #AllowAgentForwarding yes #AllowTcpForwarding yes #GatewayPorts no -#X11Forwarding yes +#X11Forwarding no #X11DisplayOffset 10 #X11UseLocalhost yes #PermitTTY yes @@ -100,7 +100,7 @@ AuthorizedKeysFile .ssh/authorized_keys #Compression delayed #ClientAliveInterval 0 #ClientAliveCountMax 3 -#UseDNS yes +#UseDNS no #PidFile /var/run/sshd.pid #MaxStartups 10:30:100 #PermitTunnel no -- 2.37.1 From 371b4c180f838a16731e2776625633bc39242728 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Mar 2016 14:42:00 +0100 Subject: [PATCH 110/310] OpenSSH ssh and sshd: Default to not sending an VersionAddendum This is also the default used by OpenSSH upstream. Obtained from: ElectroBSD --- crypto/openssh/ssh_config | 2 +- crypto/openssh/sshd_config | 2 +- crypto/openssh/version.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/crypto/openssh/ssh_config b/crypto/openssh/ssh_config index 3b2ca9aa6d8d..432c424b1f1d 100644 --- a/crypto/openssh/ssh_config +++ b/crypto/openssh/ssh_config @@ -46,4 +46,4 @@ # RekeyLimit 1G 1h # UserKnownHostsFile ~/.ssh/known_hosts.d/%k # VerifyHostKeyDNS yes -# VersionAddendum FreeBSD-20211221 +# VersionAddendum none diff --git a/crypto/openssh/sshd_config b/crypto/openssh/sshd_config index ebd1d9de81bc..e58af4482e69 100644 --- a/crypto/openssh/sshd_config +++ b/crypto/openssh/sshd_config @@ -106,7 +106,7 @@ AuthorizedKeysFile .ssh/authorized_keys #PermitTunnel no #ChrootDirectory none #UseBlacklist no -#VersionAddendum FreeBSD-20211221 +#VersionAddendum none # no default banner path #Banner none diff --git a/crypto/openssh/version.h b/crypto/openssh/version.h index 22ee862ab552..e754db5c042f 100644 --- a/crypto/openssh/version.h +++ b/crypto/openssh/version.h @@ -6,4 +6,4 @@ #define SSH_PORTABLE "p1" #define SSH_RELEASE SSH_VERSION SSH_PORTABLE -#define SSH_VERSION_FREEBSD "FreeBSD-20211221" +#define SSH_VERSION_FREEBSD "" -- 2.37.1 From 34b0403ae50ac9d06313e6e173b0ceb20e3702cd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 Feb 2016 13:15:19 +0100 Subject: [PATCH 111/310] include/Makefile: Unbreak build without atm sources Obtained from: ElectroBSD --- include/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/Makefile b/include/Makefile index 2171a241795d..ab236479f90c 100644 --- a/include/Makefile +++ b/include/Makefile @@ -54,7 +54,7 @@ LSUBDIRS= dev/acpica dev/agp dev/an dev/ciss dev/filemon dev/firewire \ geom/raid geom/raid3 geom/shsec geom/stripe geom/virstor \ net/altq \ net/route \ - netgraph/atm netgraph/netflow \ + netgraph/netflow \ netinet/cc \ netinet/netdump \ netinet/tcp_stacks \ -- 2.37.1 From 799de69f0ea1d0a6147cd38501552c49511cbfc6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Mar 2016 16:54:42 +0100 Subject: [PATCH 112/310] sys/netinet: Default to not processing IP options Obtained from: ElectroBSD --- sys/netinet/ip_options.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/netinet/ip_options.c b/sys/netinet/ip_options.c index 98bef9f844df..4b8f5f25472a 100644 --- a/sys/netinet/ip_options.c +++ b/sys/netinet/ip_options.c @@ -81,7 +81,7 @@ SYSCTL_INT(_net_inet_ip, IPCTL_ACCEPTSOURCEROUTE, accept_sourceroute, "Enable accepting source routed IP packets"); #define V_ip_acceptsourceroute VNET(ip_acceptsourceroute) -VNET_DEFINE(int, ip_doopts) = 1; /* 0 = ignore, 1 = process, 2 = reject */ +VNET_DEFINE(int, ip_doopts) = 0; /* 0 = ignore, 1 = process, 2 = reject */ SYSCTL_INT(_net_inet_ip, OID_AUTO, process_options, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_doopts), 0, "Enable IP options processing ([LS]SRR, RR, TS)"); -- 2.37.1 From 6f0d4bc1af189e54068a56d516df345ff9284450 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 25 Mar 2016 23:25:33 +0100 Subject: [PATCH 113/310] Detach freebsd-version from the build Obtained from: ElectroBSD --- bin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/bin/Makefile b/bin/Makefile index 3ad97ac8d624..d85e8019bd0e 100644 --- a/bin/Makefile +++ b/bin/Makefile @@ -15,7 +15,6 @@ SUBDIR= cat \ echo \ ed \ expr \ - freebsd-version \ getfacl \ hostname \ kenv \ -- 2.37.1 From 8f8b58f1f6711342d1b568f9964a53539dc26c42 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 15:14:25 +0200 Subject: [PATCH 114/310] geli.8: Document that setkey option -P can't be combined with -i Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index 1ad0c684642b..7fb8d5e562f8 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -609,6 +609,8 @@ option. .It Fl P Do not use a passphrase as a component of the new User Key. Cannot be combined with the +.Fl i +or .Fl J option. .El -- 2.37.1 From 7cb7a416da76a5c1e700f8fbf3f1f2fe6293004c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 15:19:53 +0200 Subject: [PATCH 115/310] geli setkey: Reject combination of -P and -i Iterations aren't used for keys without passphrase and storing a iteration count would result in a passphrase promt that can't be used to attach the provider. Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index 4c04a9256b5e..c3cb960a9fa6 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -1415,6 +1415,8 @@ eli_setkey(struct gctl_req *req) struct g_eli_metadata md; const char *prov; int nargs; + intmax_t iterations; + int nonewpassphrase; nargs = gctl_get_int(req, "nargs"); if (nargs != 1) { @@ -1423,6 +1425,14 @@ eli_setkey(struct gctl_req *req) } prov = gctl_get_ascii(req, "arg0"); + nonewpassphrase = gctl_get_int(req, "nonewpassphrase"); + iterations = gctl_get_intmax(req, "iterations"); + if (iterations != -1 && nonewpassphrase) { + gctl_error(req, + "Options -i and -P are mutually exclusive."); + return; + } + if (eli_metadata_read(req, prov, &md) == -1) return; -- 2.37.1 From 5eedd5a914ee252d433bbf25ee5ae95a57904551 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 15:24:02 +0200 Subject: [PATCH 116/310] geli setkey: Prevent passphrase removal if two keys are configured The iterations are shared by both keys and resetting it for one would break the other one as a side-effect. If this is the intention the other key can still be removed with delkey before using setkey for the remaining one. Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index c3cb960a9fa6..38945c7778a9 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -1436,6 +1436,14 @@ eli_setkey(struct gctl_req *req) if (eli_metadata_read(req, prov, &md) == -1) return; + if (nonewpassphrase && bitcount32(md.md_keys) != 1 && + md.md_iterations != -1) { + gctl_error(req, "To be able to switch from passphrase" + "-based key to passphrase-less key, only one can " + "be defined."); + return; + } + if (eli_is_attached(prov)) eli_setkey_attached(req, &md); else -- 2.37.1 From a5acd8ba690bab19a8147b3566e0ebb73e1f28cb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 15:37:03 +0200 Subject: [PATCH 117/310] geli.8: Document that setkey -P can't be used if two user keys with passphrase exist Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index 7fb8d5e562f8..a44bf269aa25 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -608,6 +608,9 @@ Cannot be combined with the option. .It Fl P Do not use a passphrase as a component of the new User Key. +This option cannot be used with the +.Cm setkey +subcommand if two User Keys with passphrase are configured. Cannot be combined with the .Fl i or -- 2.37.1 From cc9abfb7b91de8d7747077c2dc2838b5dfbc043e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 14:22:52 +0200 Subject: [PATCH 118/310] geli setkey: Fix key change from passphrase-based to passphrase-less Previously the iterations count wasn't reset resulting in a passphrase prompt that would not accept any passphrase: fk@r500 ~ $sudo geli attach -k /bpool/boot/rpool.key /dev/gpt/rpool-ada1 Enter passphrase: fk@r500 ~ $geli dump /dev/gpt/rpool-ada1 Metadata on /dev/gpt/rpool-ada1: magic: GEOM::ELI version: 7 flags: 0x0 ealgo: AES-XTS keylen: 128 provsize: 1073741824 sectorsize: 512 keys: 0x01 iterations: 463852 Salt: [...] Master Key: [...] MD5 hash: e8693274fecc65d2a13c0071fb9413b3 fk@r500 ~ $sudo geli setkey -K /bpool/boot/rpool.key -P /dev/gpt/rpool-ada1 Note, that the master key encrypted with old keys and/or passphrase may still exists in a metadata backup file. fk@r500 ~ $geli dump /dev/gpt/rpool-ada1 Metadata on /dev/gpt/rpool-ada1: magic: GEOM::ELI version: 7 flags: 0x0 ealgo: AES-XTS keylen: 128 provsize: 1073741824 sectorsize: 512 keys: 0x01 iterations: 463852 Salt: [...] Master Key: [...] MD5 hash: a443402c3b97cb37494283f8f722994d fk@r500 ~ $sudo geli detach gpt/rpool-ada1 fk@r500 ~ $sudo geli attach -k /bpool/boot/rpool.key /dev/gpt/rpool-ada1 Enter passphrase: geli: Wrong key for gpt/rpool-ada1. From userland the promt could be suppressed with "-p" (which is required for passphrase-less keys anyway), but attaching at boot time wasn't possible. PR: 196834 Reported by: Julian Hsiao Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index 38945c7778a9..7e6e6a360387 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -1291,8 +1291,16 @@ eli_setkey_attached(struct gctl_req *req, struct g_eli_metadata *md) /* Check if iterations number should be changed. */ if (val != -1) md->md_iterations = val; - else + else { old = md->md_iterations; + /* + * If the new key does not require a passphrase, + * the iterations count has to be reset to reflect + * this. + */ + if (gctl_get_int(req, "nonewpassphrase") == 1) + md->md_iterations = -1; + } /* Generate key for Master Key encryption. */ if (eli_genkey(req, md, key, true) == NULL) { @@ -1379,7 +1387,14 @@ eli_setkey_detached(struct gctl_req *req, const char *prov, return; } md->md_iterations = val; - } + } else if (gctl_get_int(req, "nonewpassphrase") == 1) { + /* + * If the new key does not require a passphrase, + * the iterations count has to be reset to reflect + * this. + */ + md->md_iterations = -1; + } mkeydst = md->md_mkeys + nkey * G_ELI_MKEYLEN; md->md_keys |= (1 << nkey); -- 2.37.1 From 9cd11a00dd5b5fb9ce8a1ce5c44fc3d7d10149fe Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 4 Apr 2016 17:37:05 +0200 Subject: [PATCH 119/310] geli.8: Document the fact that both User Keys share an iteration value ... more explicitly. Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index a44bf269aa25..dd3f059cdd13 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -825,6 +825,13 @@ the kernel from the Master Key and cached in memory. The number of Data Keys used by a given provider, and the way they are derived, depend on the GELI version and whether the provider is configured to use data authentication. +.Sh USER KEY LIMITATION +If the first User Key uses a passphrase, the second one has to use +a passphrase as well. +If the first User Key does not use a passphrase, the second one can +not use a passphrase either. +This limitation comes from the metadata format on disk which +currently only stores one iteration count for both keys. .Sh SYSCTL VARIABLES The following .Xr sysctl 8 -- 2.37.1 From f9730217c8576e5ec5e14a0c80069c511263620b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 Apr 2016 11:45:44 +0200 Subject: [PATCH 120/310] etc/rc.d: Remove a couple of requirements that don't apply to ElectroBSD Obtained from: ElectroBSD --- libexec/rc/rc.d/NETWORKING | 4 ++-- libexec/rc/rc.d/bridge | 2 +- libexec/rc/rc.d/ipfw | 1 - libexec/rc/rc.d/ipnat | 28 ---------------------------- libexec/rc/rc.d/netif | 4 ++-- libexec/rc/rc.d/routing | 2 +- libexec/rc/rc.d/securelevel | 2 +- 7 files changed, 7 insertions(+), 36 deletions(-) delete mode 100755 libexec/rc/rc.d/ipnat diff --git a/libexec/rc/rc.d/NETWORKING b/libexec/rc/rc.d/NETWORKING index 9cdb5577ed2b..1bbea5aa40b9 100755 --- a/libexec/rc/rc.d/NETWORKING +++ b/libexec/rc/rc.d/NETWORKING @@ -4,8 +4,8 @@ # # PROVIDE: NETWORKING NETWORK -# REQUIRE: netif netwait netoptions routing ppp ipfw stf -# REQUIRE: defaultroute route6d resolv bridge +# REQUIRE: netif netwait netoptions routing ipfw stf +# REQUIRE: defaultroute routed route6d resolv bridge # REQUIRE: static_arp static_ndp # This is a dummy dependency, for services which require networking diff --git a/libexec/rc/rc.d/bridge b/libexec/rc/rc.d/bridge index 95e4eb9c2fac..8631fdab2b05 100755 --- a/libexec/rc/rc.d/bridge +++ b/libexec/rc/rc.d/bridge @@ -26,7 +26,7 @@ # # PROVIDE: bridge -# REQUIRE: netif ppp stf +# REQUIRE: netif stf # KEYWORD: nojail . /etc/rc.subr diff --git a/libexec/rc/rc.d/ipfw b/libexec/rc/rc.d/ipfw index 22b65d2908cb..5944a3f99924 100755 --- a/libexec/rc/rc.d/ipfw +++ b/libexec/rc/rc.d/ipfw @@ -4,7 +4,6 @@ # # PROVIDE: ipfw -# REQUIRE: ppp # KEYWORD: nojailvnet . /etc/rc.subr diff --git a/libexec/rc/rc.d/ipnat b/libexec/rc/rc.d/ipnat deleted file mode 100755 index d4fa6b65dff4..000000000000 --- a/libexec/rc/rc.d/ipnat +++ /dev/null @@ -1,28 +0,0 @@ -#!/bin/sh -# -# $FreeBSD$ -# - -# PROVIDE: ipnat -# KEYWORD: nojailvnet - -. /etc/rc.subr - -name="ipnat" -desc="user interface to the NAT subsystem" -rcvar="ipnat_enable" -load_rc_config $name -start_cmd="ipnat_start" -stop_cmd="${ipnat_program} -F -C" -reload_cmd="${ipnat_program} -F -C -f ${ipnat_rules}" -extra_commands="reload" -required_files="${ipnat_rules}" -required_modules="ipl:ipfilter" - -ipnat_start() -{ - echo "Installing NAT rules." - ${ipnat_program} -CF -f ${ipnat_rules} ${ipnat_flags} -} - -run_rc_command "$1" diff --git a/libexec/rc/rc.d/netif b/libexec/rc/rc.d/netif index 3dbb3e1a9588..b0d405b8cd45 100755 --- a/libexec/rc/rc.d/netif +++ b/libexec/rc/rc.d/netif @@ -26,9 +26,9 @@ # # PROVIDE: netif -# REQUIRE: FILESYSTEMS iovctl serial sppp sysctl +# REQUIRE: FILESYSTEMS iovctl serial sysctl # REQUIRE: hostid -# KEYWORD: nojailvnet +# KEYWORD: nojailvnet shutdown . /etc/rc.subr . /etc/network.subr diff --git a/libexec/rc/rc.d/routing b/libexec/rc/rc.d/routing index 043c5b15fbaa..36e57aed4843 100755 --- a/libexec/rc/rc.d/routing +++ b/libexec/rc/rc.d/routing @@ -6,7 +6,7 @@ # # PROVIDE: routing -# REQUIRE: netif ppp stf +# REQUIRE: netif stf # KEYWORD: nojailvnet . /etc/rc.subr diff --git a/libexec/rc/rc.d/securelevel b/libexec/rc/rc.d/securelevel index 8bb09dd920bd..f25841afcffd 100755 --- a/libexec/rc/rc.d/securelevel +++ b/libexec/rc/rc.d/securelevel @@ -4,7 +4,7 @@ # # PROVIDE: securelevel -# REQUIRE: adjkerntz ipfw pf sysctl_lastload +# REQUIRE: adjkerntz pf sysctl_lastload . /etc/rc.subr -- 2.37.1 From 05f5a2205acdac32e720f0558b0d8dd20ca3817a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 25 Apr 2016 12:48:33 +0200 Subject: [PATCH 121/310] rc.d: Change geli_autodetach default to 'NO' ... as autodetach leads to panics when scrubbing ZFS pools with more than one device. For details see: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=117158 Obtained from: ElectroBSD --- libexec/rc/rc.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) mode change 100644 => 100755 libexec/rc/rc.conf diff --git a/libexec/rc/rc.conf b/libexec/rc/rc.conf old mode 100644 new mode 100755 index 422c2efa9b59..b034ac46e48f --- a/libexec/rc/rc.conf +++ b/libexec/rc/rc.conf @@ -92,7 +92,7 @@ geli_groups="" # List of groups containing devices to automatically geli_tries="" # Number of times to attempt attaching geli device. # If empty, kern.geom.eli.tries will be used. geli_default_flags="" # Default flags for geli(8). -geli_autodetach="YES" # Automatically detach on last close. +geli_autodetach="NO" # Automatically detach on last close. # Providers are marked as such when all file systems are # mounted. # Example use. -- 2.37.1 From 1ba948c9723a85644cae82988ea3f3449758eb32 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 14 May 2016 22:05:48 +0200 Subject: [PATCH 122/310] disklatency: Skip invalid io::done probes .. until the cause has been analyzed and fixed Obtained from: ElectroBSD --- share/dtrace/disklatency | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/dtrace/disklatency b/share/dtrace/disklatency index 97ef87a07980..21ff6f719053 100755 --- a/share/dtrace/disklatency +++ b/share/dtrace/disklatency @@ -42,7 +42,7 @@ io:::start } io:::done -/this->start = start_time[arg0]/ +/(this->start = start_time[arg0]) && (args[1]->unit_number != -1)/ { this->delta = (timestamp - this->start) / 1000; @q[args[1]->device_name, args[1]->unit_number] = -- 2.37.1 From a3207108aa34579dce8f80222f50de47e430bbdc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 14 May 2016 22:06:51 +0200 Subject: [PATCH 123/310] disklatency: Reformat output to print device name and unit together While at it, add a delimiter. Obtained from: ElectroBSD --- share/dtrace/disklatency | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/share/dtrace/disklatency b/share/dtrace/disklatency index 21ff6f719053..930be21d0afb 100755 --- a/share/dtrace/disklatency +++ b/share/dtrace/disklatency @@ -55,10 +55,11 @@ io:::done tick-10s { - printa(" %s (%d), us:\n%@d\n", @q); - printa("max%s (%d), us:\n%@d\n", @max); - printa("avg%s (%d), us:\n%@d\n", @avg); - printa("stddev%s (%d), us:\n%@d\n", @stddev); + printf("---------------------------------------------\n"); + printa("%s%d, us:\n%@d\n", @q); + printa("%s%d max, us:\n%@d\n", @max); + printa("%s%d avg, us:\n%@d\n", @avg); + printa("%s%d stdev, us:\n%@d\n", @stddev); clear(@q); clear(@max); -- 2.37.1 From c09e3f37159c0b5fbed367ed3a46f12ebbf7ec84 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 15 May 2016 09:30:31 +0200 Subject: [PATCH 124/310] disklatency: Use trunc() instead of clear() ... so only latency stats for disks with activity in the relevant intervals are shown. Obtained from: ElectroBSD --- share/dtrace/disklatency | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/share/dtrace/disklatency b/share/dtrace/disklatency index 930be21d0afb..f4081c14bc84 100755 --- a/share/dtrace/disklatency +++ b/share/dtrace/disklatency @@ -61,8 +61,8 @@ tick-10s printa("%s%d avg, us:\n%@d\n", @avg); printa("%s%d stdev, us:\n%@d\n", @stddev); - clear(@q); - clear(@max); - clear(@avg); - clear(@stddev); + trunc(@q); + trunc(@max); + trunc(@avg); + trunc(@stddev); } -- 2.37.1 From 933599fd33a9c9e42fd20d83e15b3995c7e4f0da Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 1 May 2016 16:27:55 +0200 Subject: [PATCH 125/310] vmstat: Prevent output truncation when piping zone statistics ... into other programs. Apparently libxo (currently?) can't be trusted to flush its internal buffer before it's full, triggering the flushing manually works around this. Before (sometimes): fk@r500 ~ $vmstat -z | wc -c 8192 After (consistently): fk@r500 ~ $vmstat -z | wc -c 24156 Obtained from: ElectroBSD PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206128 --- usr.bin/vmstat/vmstat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/usr.bin/vmstat/vmstat.c b/usr.bin/vmstat/vmstat.c index 403dc6e2a054..4c099eff53c6 100644 --- a/usr.bin/vmstat/vmstat.c +++ b/usr.bin/vmstat/vmstat.c @@ -1521,6 +1521,7 @@ domemstat_zone(void) (uintmax_t)memstat_get_sleeps(mtp), (uintmax_t)memstat_get_xdomain(mtp)); xo_close_instance("zone"); + xo_flush(); } memstat_mtl_free(mtlp); xo_close_list("zone"); -- 2.37.1 From 8cb474028ac9d79c6d8241ce2793e0bf43f7c0e0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 27 May 2016 18:55:19 +0200 Subject: [PATCH 126/310] atkbdc: Work around (apparently) broken mouse reinitialization ... by increasing pkterrthresh to 10. Previously moving the mouse while loading DTrace modules, for example by running a DTrace script without having loaded the modules manually first, would reliably disables the mouse (track point) until reboot (on a ThinkPad R500). With debug.psm.loglevel=5 the logs showed: Nov 25 13:38:06 r500 kernel: [132] psmintr: 28 03 fe 01 ff 00 Nov 25 13:38:06 r500 kernel: [132] psmintr: 28 02 ff 01 ff 00 Nov 25 13:38:06 r500 kernel: [132] psmintr: 28 03 fe 02 ff 00 Nov 25 13:38:06 r500 kernel: [132] psmintr: 08 02 00 03 fe 00 Nov 25 13:38:06 r500 kernel: [132] psmintr: 28 01 ff 03 fe 00 Nov 25 13:38:07 r500 kernel: [132] psmintr: 18 fe 00 03 ff 00 Nov 25 13:38:08 r500 sudo: fk : TTY=ttyv0 ; PWD=/home/fk ; USER=root ; COMMAND=/usr/share/dtrace/toolkit/execsnoop Nov 25 13:38:08 r500 kernel: [134] psm0: lost interrupt? Nov 25 13:38:09 r500 kernel: [134] psmintr: 08 00 01 03 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 08 00 01 04 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 ff 01 04 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 ff 02 05 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 ff 02 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fe 03 05 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fe 04 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fd 05 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fd 05 03 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fe 03 05 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fd 03 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fa 06 05 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fc 04 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fa 03 04 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 f9 05 07 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 f7 03 05 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 f9 04 06 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 f8 02 04 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fb 01 04 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fb 01 02 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 ff 00 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 18 ff 00 02 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: c8 d3 0b 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: out of sync (00c0 != 0000) 881 cmds since last error. Nov 25 13:38:11 r500 kernel: [136] psmintr: discard a byte (1) Nov 25 13:38:11 r500 kernel: [136] psmintr: d3 0b 08 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: out of sync (00c0 != 0000) 0 cmds since last error. Nov 25 13:38:11 r500 kernel: [136] psmintr: discard a byte (2) Nov 25 13:38:11 r500 kernel: [136] psmintr: 0b 08 4b 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 3c 08 0c 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0f 08 0c 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 10 08 0b 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 10 08 0a 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 12 08 08 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 10 08 08 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 12 08 08 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 13 08 05 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0f 08 05 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 13 08 04 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 13 08 01 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0b 08 00 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0a 18 ff 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0f 18 fd 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0e 18 fc 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 08 18 fc 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0a 18 fc 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 05 18 fb 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 05 18 fa 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 05 18 f5 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 09 18 f2 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 08 18 f2 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 05 18 f0 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 06 18 f2 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 03 18 ee 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 02 18 f3 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 01 18 ef 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 00 38 ed 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: ff 38 ee 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: out of sync (00c0 != 0000) 0 cmds since last error. Nov 25 13:38:11 r500 kernel: [136] psmintr: reset the mouse. Nov 25 13:38:12 r500 kernel: [137] psm0: current command byte: 0047 (reinitialize). Nov 25 13:38:12 r500 kernel: [137] psm: DISABLE_DEV return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: ENABLE_DEV return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: DISABLE_DEV return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: SET_SAMPLING_RATE (100) 00fa Nov 25 13:38:12 r500 kernel: [137] psm: SET_RESOLUTION (2) 00fa Nov 25 13:38:12 r500 kernel: [137] psm: SET_SCALING11 return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: SET_STREAM_MODE return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: SEND_AUX_DEV_STATUS return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: status 00 02 64 Nov 25 13:38:12 r500 kernel: [137] psm: ENABLE_DEV return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: SEND_AUX_DEV_STATUS return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: status 20 02 64 Nov 25 13:38:15 r500 kernel: [140] psm0: lost interrupt? Nov 25 13:38:16 r500 kernel: [141] psm0: lost interrupt? Nov 25 13:38:17 r500 kernel: [142] psm0: lost interrupt? Nov 25 13:38:18 r500 kernel: [143] psm0: lost interrupt? Nov 25 13:38:19 r500 kernel: [144] psm0: lost interrupt? Nov 25 13:38:20 r500 kernel: [145] psm0: lost interrupt? Nov 25 13:38:21 r500 kernel: [147] psm0: lost interrupt? Nov 25 13:38:22 r500 kernel: [148] psm0: lost interrupt? Nov 25 13:38:23 r500 kernel: [149] psm0: lost interrupt? Nov 25 13:38:24 r500 kernel: [150] psm0: lost interrupt? Nov 25 13:38:25 r500 kernel: [151] psm0: lost interrupt? Nov 25 13:38:26 r500 kernel: [152] psm0: lost interrupt? Nov 25 13:38:27 r500 kernel: [153] psm0: lost interrupt? Nov 25 13:38:29 r500 kernel: [154] psm0: lost interrupt? Nov 25 13:38:30 r500 kernel: [155] psm0: lost interrupt? Nov 25 13:38:31 r500 kernel: [156] psm0: lost interrupt? Nov 25 13:38:32 r500 kernel: [157] psm0: lost interrupt? Nov 25 13:38:33 r500 kernel: [158] psm0: lost interrupt? Nov 25 13:38:34 r500 kernel: [159] psm0: lost interrupt? Nov 25 13:38:35 r500 kernel: [160] psm0: lost interrupt? Nov 25 13:38:36 r500 kernel: [161] psm0: lost interrupt? Nov 25 13:38:37 r500 kernel: [162] psm0: lost interrupt? Nov 25 13:38:38 r500 kernel: [163] psm0: lost interrupt? Nov 25 13:38:39 r500 kernel: [164] psm0: lost interrupt? Nov 25 13:38:40 r500 kernel: [165] psm0: lost interrupt? After the "reset" the mouse cursor no longer moved and rebooting seemed to be the only "cure". Setting debug.psm.pkterrthresh=10 seems to work around the issue, so does not moving the cursor until the modules are loaded. Mouse resets for other reasons seemed to occasionally render the mouse useless as well, but triggering the problem by loading DTrace modules is the only reproducible method currently known. Obtained from: ElectroBSD PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221305 PR submission date: 2017-08-07 --- sys/dev/atkbdc/psm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/atkbdc/psm.c b/sys/dev/atkbdc/psm.c index 9b47a7a6b0c5..48570f864f16 100644 --- a/sys/dev/atkbdc/psm.c +++ b/sys/dev/atkbdc/psm.c @@ -2965,7 +2965,7 @@ SYSCTL_INT(_debug_psm, OID_AUTO, secs, CTLFLAG_RW, &psmsecs, 0, static int psmusecs = 500000; SYSCTL_INT(_debug_psm, OID_AUTO, usecs, CTLFLAG_RW, &psmusecs, 0, "Microseconds to add to psmsecs"); -static int pkterrthresh = 2; +static int pkterrthresh = 10; SYSCTL_INT(_debug_psm, OID_AUTO, pkterrthresh, CTLFLAG_RW, &pkterrthresh, 0, "Number of error packets allowed before reinitializing the mouse"); -- 2.37.1 From c0251eb9ff1aac406b0eb1fe7102d5c18bff9a81 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 24 Jun 2016 14:39:28 +0200 Subject: [PATCH 127/310] sys/amd64/conf/ELECTRO_BLOAT: Remove COMPAT support for FreeBSD versions before 10 Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 5 ----- 1 file changed, 5 deletions(-) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index a61f4e5eee3a..3bd60d2df610 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -48,11 +48,6 @@ options GEOM_PART_GPT # GUID Partition Tables. options GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization options COMPAT_FREEBSD32 # Compatible with i386 binaries -options COMPAT_FREEBSD4 # Compatible with FreeBSD4 -options COMPAT_FREEBSD5 # Compatible with FreeBSD5 -options COMPAT_FREEBSD6 # Compatible with FreeBSD6 -options COMPAT_FREEBSD7 # Compatible with FreeBSD7 -options COMPAT_FREEBSD9 # Compatible with FreeBSD9 options COMPAT_FREEBSD10 # Compatible with FreeBSD10 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support -- 2.37.1 From efef45021bdc82e9e67814e903f614c2b3b5c9a1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 5 May 2016 13:59:47 +0200 Subject: [PATCH 128/310] share/skel: Detach skelleton files for csh and rsh from the build Obtained from: ElectroBSD --- share/skel/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/skel/Makefile b/share/skel/Makefile index 9ad66cff61e8..784a9adeaf84 100644 --- a/share/skel/Makefile +++ b/share/skel/Makefile @@ -1,7 +1,7 @@ # @(#)Makefile 8.1 (Berkeley) 6/8/93 # $FreeBSD$ -FILES= dot.cshrc dot.login dot.login_conf dot.mailrc dot.profile \ +FILES= dot.login dot.login_conf dot.mailrc dot.profile \ dot.shrc dot.mail_aliases FILESDIR= ${SHAREDIR}/skel FILESMODE= 0644 -- 2.37.1 From 32f0d4e7b4edf084f9502d499b2eb98a9e1d9b21 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 20 Sep 2015 19:53:09 +0200 Subject: [PATCH 129/310] top: Show ZFS ARC target size Obtained from: ElectroBSD --- usr.bin/top/machine.c | 4 +++- usr.bin/top/top.1 | 8 +++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/usr.bin/top/machine.c b/usr.bin/top/machine.c index 315c8be1a4c2..0eb8fce5e94a 100644 --- a/usr.bin/top/machine.c +++ b/usr.bin/top/machine.c @@ -133,7 +133,7 @@ static const char *memorynames[] = { static int memory_stats[nitems(memorynames)]; static const char *arcnames[] = { - "K Total, ", "K MFU, ", "K MRU, ", "K Anon, ", "K Header, ", "K Other", + "K Total, ", "K MFU, ", "K MRU, ", "K Anon, ", "K Header, ", "K Other, ", "K Target", NULL }; static int arc_stats[nitems(arcnames)]; @@ -545,6 +545,8 @@ get_system_info(struct system_info *si) arc_stats[5] += arc_stat >> 10; GETSYSCTL("kstat.zfs.misc.arcstats.dbuf_size", arc_stat); arc_stats[5] += arc_stat >> 10; + GETSYSCTL("kstat.zfs.misc.arcstats.c", arc_stat); + arc_stats[6] = arc_stat >> 10; si->arc = arc_stats; } if (carc_enabled) { diff --git a/usr.bin/top/top.1 b/usr.bin/top/top.1 index 3aff7012f3d8..c1a86b01c645 100644 --- a/usr.bin/top/top.1 +++ b/usr.bin/top/top.1 @@ -445,7 +445,7 @@ are not. .Sh DESCRIPTION OF MEMORY .Bd -literal Mem: 61M Active, 86M Inact, 368K Laundry, 22G Wired, 102G Free -ARC: 15G Total, 9303M MFU, 6155M MRU, 1464K Anon, 98M Header, 35M Other +ARC: 15G Total, 9303M MFU, 6155M MRU, 1464K Anon, 98M Header, 35M Other, 15G Target 15G Compressed, 27G Uncompressed, 1.75:1 Ratio, 174M Overhead Swap: 4096M Total, 532M Free, 13% Inuse, 80K In, 104K Out .Ed @@ -480,6 +480,12 @@ number of ARC bytes holding in flight data number of ARC bytes holding headers .It Em Other miscellaneous ARC bytes +.B Target: +ARC target size, that is the total amount of memory +the ARC considers usable for itself. If it's not equal +to the total size, the ARC will shrink or grow to reach +the target. +.TP .It Em Compressed bytes of memory used by ARC caches .It Em Uncompressed -- 2.37.1 From edd86ed5bd8245d8a1ee74483ffe90d21a92333c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 25 Oct 2015 19:49:47 +0100 Subject: [PATCH 130/310] ZFS dsl_scan_visitds(): Don't panic if a device disappears while scrubbing Prevents: Unread portion of the kernel message buffer: [4299] panic: solaris assert: dmu_objset_find_dp(dp, dp->dp_root_dir_obj, enqueue_clones_cb, &eca, (1<<1)) == 0 (0x6 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c, line: 1130 [4299] cpuid = 1 [4299] KDB: stack backtrace: [4299] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00949ed310 [4299] vpanic() at vpanic+0x182/frame 0xfffffe00949ed390 [4299] panic() at panic+0x43/frame 0xfffffe00949ed3f0 [4299] zfs_kmem_alloc() at zfs_kmem_alloc/frame 0xfffffe00949ed440 [4299] dsl_scan_visitds() at dsl_scan_visitds+0x551/frame 0xfffffe00949ed570 [4299] dsl_scan_visit() at dsl_scan_visit+0x22e/frame 0xfffffe00949ed790 [4299] dsl_scan_sync() at dsl_scan_sync+0x9da/frame 0xfffffe00949ed920 [4299] spa_sync() at spa_sync+0x564/frame 0xfffffe00949eda90 [4299] txg_sync_thread() at txg_sync_thread+0x3f1/frame 0xfffffe00949edbb0 [4299] fork_exit() at fork_exit+0x9c/frame 0xfffffe00949edbf0 [4299] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00949edbf0 [4299] --- trap 0, rip = 0, rsp = 0, rbp = 0 --- [4299] KDB: enter: panic Note that this is not the only place where dmu_objset_find_dp() failures lead to panics and ENXIO probably isn't the only return code we have to expect anyway. Obtained from: ElectroBSD --- sys/contrib/openzfs/module/zfs/dsl_scan.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sys/contrib/openzfs/module/zfs/dsl_scan.c b/sys/contrib/openzfs/module/zfs/dsl_scan.c index d7050487ff82..8b3f4f5108a0 100644 --- a/sys/contrib/openzfs/module/zfs/dsl_scan.c +++ b/sys/contrib/openzfs/module/zfs/dsl_scan.c @@ -2511,9 +2511,11 @@ dsl_scan_visitds(dsl_scan_t *scn, uint64_t dsobj, dmu_tx_t *tx) } zap_cursor_fini(&zc); } else { - VERIFY0(dmu_objset_find_dp(dp, dp->dp_root_dir_obj, + int error; + error = dmu_objset_find_dp(dp, dp->dp_root_dir_obj, enqueue_clones_cb, &ds->ds_object, - DS_FIND_CHILDREN)); + DS_FIND_CHILDREN); + VERIFY(error == 0 || error == ENXIO); } } -- 2.37.1 From 4dc9a02dcfd99dc5319c45811c4337765d083fbd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 30 Dec 2016 16:54:28 +0100 Subject: [PATCH 131/310] g_dev_orphan(): Return early if the device is already gone Supposed to fix a panic that could occur while running "cdcontrol eject" after using the physical ejection key on the device: Unread portion of the kernel message buffer: stack pointer = 0x28:0xfffffe01eba0a9e0 frame pointer = 0x28:0xfffffe01eba0a9f0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 13 (g_event) trap number = 12 panic: page fault cpuid = 3 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01eba0a560 vpanic() at vpanic+0x182/frame 0xfffffe01eba0a5e0 panic() at panic+0x43/frame 0xfffffe01eba0a640 trap_fatal() at trap_fatal+0x331/frame 0xfffffe01eba0a6a0 trap_pfault() at trap_pfault+0x1e3/frame 0xfffffe01eba0a700 trap() at trap+0x273/frame 0xfffffe01eba0a910 calltrap() at calltrap+0x8/frame 0xfffffe01eba0a910 --- trap 0xc, rip = 0xffffffff80500fde, rsp = 0xfffffe01eba0a9e0, rbp = 0xfffffe01eba0a9f0 --- g_dev_orphan() at g_dev_orphan+0x2e/frame 0xfffffe01eba0a9f0 g_resize_provider_event() at g_resize_provider_event+0x71/frame 0xfffffe01eba0aa20 g_run_events() at g_run_events+0x20e/frame 0xfffffe01eba0aa70 fork_exit() at fork_exit+0x85/frame 0xfffffe01eba0aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01eba0aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 3h17m41s Dumping 1120 out of 8055 MB:..2%..12%..22%..32%..42%..52%..62%..72%..82%..92% Reading symbols from /usr/lib/debug/boot/kernel/zfs.ko.debug...done. [...] Loaded symbols for /usr/lib/debug/boot/kernel/iicbb.ko.debug #0 doadump (textdump=1) at pcpu.h:222 222 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump (textdump=1) at pcpu.h:222 #1 0xffffffff805cce3e in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff805cd40b in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff805cd243 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:690 #4 0xffffffff808eece1 in trap_fatal (frame=0xfffffe01eba0a920, eva=8) at /usr/src/sys/amd64/amd64/trap.c:801 #5 0xffffffff808eeed3 in trap_pfault (frame=0xfffffe01eba0a920, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:658 #6 0xffffffff808ee4d3 in trap (frame=0xfffffe01eba0a920) at /usr/src/sys/amd64/amd64/trap.c:421 #7 0xffffffff808d2701 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #8 0xffffffff80500fde in g_dev_orphan (cp=0xfffff800069e6280) at /usr/src/sys/geom/geom_dev.c:754 #9 0xffffffff80509ff1 in g_resize_provider_event (arg=, flag=) at /usr/src/sys/geom/geom_subr.c:631 #10 0xffffffff80504f1e in g_run_events () at /usr/src/sys/geom/geom_event.c:264 #11 0xffffffff805830e5 in fork_exit (callout=0xffffffff805079c0 , arg=0x0, frame=0xfffffe01eba0aac0) at /usr/src/sys/kern/kern_fork.c:1040 #12 0xffffffff808d2c3e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611 #13 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) f 8 #8 0xffffffff80500fde in g_dev_orphan (cp=0xfffff800069e6280) at /usr/src/sys/geom/geom_dev.c:754 754 g_trace(G_T_TOPOLOGY, "g_dev_orphan(%p(%s))", cp, cp->geom->name); (kgdb) p cp $1 = (struct g_consumer *) 0xfffff800069e6280 (kgdb) p cp->geom $2 = (struct g_geom *) 0xfffff801acd6d100 (kgdb) p cp->geom->name $3 = 0xfffff8007479bf60 "cd0" (kgdb) f 8 #8 0xffffffff80500fde in g_dev_orphan (cp=0xfffff800069e6280) at /usr/src/sys/geom/geom_dev.c:754 754 g_trace(G_T_TOPOLOGY, "g_dev_orphan(%p(%s))", cp, cp->geom->name); (kgdb) l 754 g_trace(G_T_TOPOLOGY, "g_dev_orphan(%p(%s))", cp, cp->geom->name); 755 756 /* Reset any dump-area set on this device */ 757 if (dev->si_flags & SI_DUMPDEV) 758 (void)set_dumper(NULL, NULL, curthread); 759 760 /* Destroy the struct cdev *so we get no more requests */ 761 destroy_dev_sched_cb(dev, g_dev_callback, cp); 762 } 763 (kgdb) p dev->si_flags Cannot access memory at address 0x8 (kgdb) p dev $4 = (struct cdev *) 0x0 Last message before the panic: Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:33:49 t520 kernel: ahcich1: Timeout on slot 8 port 0 Dec 30 16:33:49 t520 kernel: ahcich1: is 00000000 cs 00000100 ss 00000000 rs 00000100 tfd d0 serr 00000000 cmd 000cc817 Dec 30 16:33:49 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 eb 00 00 01 00 Dec 30 16:33:49 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: Command timeout Dec 30 16:33:49 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 eb 00 00 01 00 Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: MEDIUM ERROR asc:10,0 (ID CRC or ECC error) Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 ad 00 00 04 00 Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: MEDIUM ERROR asc:10,0 (ID CRC or ECC error) Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 ad 00 00 04 00 Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: MEDIUM ERROR asc:10,0 (ID CRC or ECC error) Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 eb 00 00 01 00 Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): Error 6, Unretryable error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x6 back Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 ea 00 00 01 00 Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): Error 6, Unretryable error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x6 back Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 ea 00 00 01 00 Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): Error 6, Unretryable error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x6 back Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 eb 00 00 01 00 Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): Error 6, Unretryable error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x6 back PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215856 PR submission date: 2017-01-07 Obtained from: ElectroBSD --- sys/geom/geom_dev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sys/geom/geom_dev.c b/sys/geom/geom_dev.c index e52f8b8cccc2..74ed40308069 100644 --- a/sys/geom/geom_dev.c +++ b/sys/geom/geom_dev.c @@ -872,6 +872,9 @@ g_dev_orphan(struct g_consumer *cp) dev = sc->sc_dev; g_trace(G_T_TOPOLOGY, "g_dev_orphan(%p(%s))", cp, cp->geom->name); + if (dev == NULL) + return; + /* Reset any dump-area set on this device */ if (dev->si_flags & SI_DUMPDEV) { struct diocskerneldump_arg kda; -- 2.37.1 From e8b5f60e4f5bf34a4884137994c68638cf60041b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 25 Jun 2015 17:26:59 +0200 Subject: [PATCH 132/310] Add reproduce.sh which makes reproducing ElectroBSD more convenient Squashed commits worth mentioning: - Add -j option to overwrite the maximum number of make jobs. - Assert that the source directory is untainted (according to strip-freebsd.sh) and add -a flag to remove offending files. - Allow to resume a build by using the -r flag. - Add -p option to change the prefix for the source and object directories. This could be useful when building ElectroBSD as port. - Add a -d option to specifiy the directory to move the produced distfiles into. - Use time's -h flag to get more pleasant number. Obtained from: ElectroBSD --- reproduce.sh | 260 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 260 insertions(+) create mode 100755 reproduce.sh diff --git a/reproduce.sh b/reproduce.sh new file mode 100755 index 000000000000..93a525772a9d --- /dev/null +++ b/reproduce.sh @@ -0,0 +1,260 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2015-2016 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## + +# reproduce.sh +# +# Script to make reproducing an ElectroBSD build more convenient. +# Before using it, make sure BUILD and EPOCH contain values other +# than __BUILD__ and __EPOCH__, either by editing the script or +# by putting them in a configuration file that is speficied with +# the -f option. + +# These variables have to be set to the values used for the build +# that is supposed to be reproduced. ${SRC_DIR} must contain the +# matching sources! +BUILD=__BUILD__ +EPOCH=__EPOCH__ + +# This is just a suggestion, feel free to overwrite it with the -j option. +MAX_MAKE_JOBS="${MAX_MAKE_JOBS-2}" + +# Prefix to use for SRC_DIR and MAKEOBJDIRPREFIX +DIRECTORY_PREFIX="${DIRECTORY_PREFIX-/}" + +# Currently somewhat hardcoded. +SRC_DIR="${DIRECTORY_PREFIX}usr/src" +MAKEOBJDIRPREFIX="${DIRECTORY_PREFIX}usr/obj" + +# Config file location when -f isn't specified +OPTIONAL_CONFIG_FILE="${SRC_DIR}/reproduce.conf" + +# Make sure we respawn with the same script, even if it is located +# outside the SRC_DIR and called with a relative path. +REPRODUCE_SH="$(realpath "${0}")" + +# When set to true, existing object files will be reused. +# If the source files changed, the result will not be reproducible! +RESUME_BUILD="${RESUME_BUILD-false}" + +# Number of threads to use when compressing with xz. +# +# The upstream default is 0 (auto-tune) which results +# in unreproducible results when using systems with a +# different number of cores. +XZ_THREADS="${XZ_THREADS-2}" + +# Used by announce_status() to create a timestamp prefix +LOG_TIMESTAMP_FORMAT='+%Y-%m-%d %H:%M:%S' + +announce_status() { + local msg \ + timestamp + + msg="${*}" + timestamp=$(date "${LOG_TIMESTAMP_FORMAT}") + + echo "${timestamp}: ${msg}" +} + +reproduce_all_the_things() { + local \ + f release_dir + + # In a perfect world the umask shouldn't matter here but + # in the real world it does for the single file /etc/rc.local + # on the memstick. + umask 022 + + release_dir="$(make -C "${SRC_DIR}/release" -V .OBJDIR)" + if [ -z "${release_dir}" ]; then + announce_status "Failed to get release_dir through .OBJDIR ..." + return 1 + fi + announce_status "Using release dir ${release_dir} ..." + + if "${RESUME_BUILD}"; then + announce_status "Resuming ..." + export KERNFAST=1 + export NO_CLEAN=1 + fi + export MAKEOBJDIRPREFIX + + # We build the world first, so the kernel is built + # with a freshly built toolchain. + announce_status "Starting to build the world" + time -h -l make -j${MAX_MAKE_JOBS} buildworld || return 1 + + announce_status "Starting to build the kernel" + time -h -l make buildkernel || return 1 + + # Make sure obj files aren't dumped in ${SRC_DIR} + mkdir -p "${release_dir}" || return 1 + + if ! "${RESUME_BUILD}"; then + announce_status "Starting to clean the release dir" + make -C "${SRC_DIR}/release" clean + fi + announce_status "Starting to build the release" + time -h -l make -C "${SRC_DIR}/release" \ + memstick XZ_THREADS=${XZ_THREADS} NO_FSCHG="yes" || return 1 + + announce_status "Done with release memstick for ${REPRO_SEED}" + + cd "${release_dir}" || return 1 + + mkdir -p "${DISTFILE_DIR}" || return 1 + for f in *.txz MANIFEST; do + mv "${f}" "${DISTFILE_DIR}/" || return 1 + done + mv memstick.img "${DISTFILE_DIR}/${BUILD}.img" || return 1 + + announce_status "Created files copied to ${DISTFILE_DIR}" + sha256 "${DISTFILE_DIR}"/* +} + +assert_untainted_source_tree() { + local auto_untaint \ + untaint_commands + + auto_untaint="${1}" + + untaint_commands="$(sh ./release/scripts/strip-freebsd.sh .)" + if [ -n "${untaint_commands}" ] ; then + if $auto_untaint; then + echo "Auto untainting $(pwd)" + echo "${untaint_commands}" | sh -x || return 1 + else + echo "${SRC_DIR} is tainted. Use -a flag to auto-untaint it." + return 1 + fi + fi + return 0 +} + +respawn_with_clean_environment() { + exec env -i PATH="/sbin:/bin:/usr/sbin:/usr/bin" HOME="/root" \ + LC_COLLATE=C SHELL=/bin/sh ALREADY_RESPAWNED=1 \ + DISTFILE_DIR="${DISTFILE_DIR}" \ + MAX_MAKE_JOBS="${MAX_MAKE_JOBS}" RESUME_BUILD="${RESUME_BUILD}" \ + DIRECTORY_PREFIX="${DIRECTORY_PREFIX}" XZ_THREADS=${XZ_THREADS} \ + REPRO_SEED="${REPRO_SEED}" SRCCONF=/dev/null /bin/sh "${REPRODUCE_SH}" +} + +main() { + local args \ + auto_untaint config_file fake_user dry_run + + fake_user=elektropunker + + auto_untaint=false + dry_run=false + config_file="${OPTIONAL_CONFIG_FILE}" + + args=$(getopt ad:f:j:npr $*) + if [ $? -ne 0 ]; then + echo 'You are doing it wrong: Invalid flag specified' + exit 2 + fi + set -- ${args} + while true; do + case "$1" in + -a) + shift + auto_untaint="true" + ;; + -d) + shift; + DISTFILE_DIR="${1}" + shift; + ;; + -j) + shift; + MAX_MAKE_JOBS="${1}" + shift; + ;; + -f) + shift; + config_file="${1}" + shift; + if [ ! -f "${config_file}" ]; then + echo "Config file ${config_file} does not exist" + exit 2 + fi + ;; + -n) + dry_run=true + shift + ;; + -p) + shift + DIRECTORY_PREFIX="${1}" + shift; + if [ ! -d "${DIRECTORY_PREFIX}" ]; then + echo "Directory ${DIRECTORY_PREFIX} specified with -p does not exist" + exit 2 + fi + # This is only needed for the cd below + SRC_DIR="${DIRECTORY_PREFIX}${SRC_DIR}" + ;; + -r) + shift + RESUME_BUILD=true + ;; + --) + shift; break + ;; + esac + done + + if [ -f "${config_file}" ]; then + announce_status "Reading config from ${config_file}" + . "${config_file}" || exit 2 + fi + + if [ "${BUILD}" = "__BUILD__" ]; then + announce_status "BUILD not set" + return 1 + fi + if [ "${EPOCH}" = "__EPOCH__" ]; then + announce_status "EPOCH not set" + return 1 + fi + if [ -n "${ALREADY_RESPAWNED}" -a "${ALREADY_RESPAWNED}" = 1 ]; then + if [ -z "${REPRO_SEED}" ]; then + announce_status "Respawned with REPRO_SEED unset" + return 1 + fi + reproduce_all_the_things + return + fi + + export REPRO_SEED="${fake_user}:${BUILD}:${EPOCH}" + export DISTFILE_DIR="${DISTFILE_DIR-${MAKEOBJDIRPREFIX}${SRC_DIR}/${BUILD}-$(date +%Y-%m-%d-%H:%M)}" + + announce_status "REPRO_SEED=${REPRO_SEED}" + + cd "${SRC_DIR}" || return 1 + + assert_untainted_source_tree "${auto_untaint}" || return 1 + + if ! $dry_run; then + respawn_with_clean_environment + fi +} + +main "${@}" -- 2.37.1 From efc0352f0a5fe77e3982ea3d14a61ef72bc35679 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 18 Apr 2015 19:38:07 +0200 Subject: [PATCH 133/310] gmountver.8: Note that GEOM mount verification has dangerous bugs Obtained from: ElectroBSD --- lib/geom/mountver/gmountver.8 | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/lib/geom/mountver/gmountver.8 b/lib/geom/mountver/gmountver.8 index 96bf82ee30c0..8f3e62200236 100644 --- a/lib/geom/mountver/gmountver.8 +++ b/lib/geom/mountver/gmountver.8 @@ -60,6 +60,8 @@ got disconnected - it queues all the I/O requests and waits for the provider to reappear. When that happens, it attaches to it and sends the queued requests. .Pp +At least that's the theory, please note the BUGS section. +.Pp The first argument to .Nm indicates an action to be performed: @@ -118,6 +120,13 @@ If set to 0, .Nm will reattach to the device even if the device reports a different disk ID. .El +.Sh BUGS +The mount verification GEOM class can stall all the disk I/O instead +of just the device it is configured for. +The disk identification check currently has to be turned off for the class +to actually attach reappearing providers. +Obviously this is dangerous. +.El .Sh EXIT STATUS Exit status is 0 on success, and 1 if the command fails. .Sh SEE ALSO -- 2.37.1 From 68667714f53f7d0833ab071ff0c2c2d4cbcfa44f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 28 Jan 2017 14:41:17 +0100 Subject: [PATCH 134/310] Prevent inlining of g_mountver_ident_matches() so the taste failures can be debugged Obtained from: ElectroBSD --- sys/geom/mountver/g_mountver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/geom/mountver/g_mountver.c b/sys/geom/mountver/g_mountver.c index 9703f1d02529..39ffd7ab27ea 100644 --- a/sys/geom/mountver/g_mountver.c +++ b/sys/geom/mountver/g_mountver.c @@ -518,7 +518,7 @@ g_mountver_resize(struct g_consumer *cp) g_resize_provider(pp, cp->provider->mediasize); } -static int +static __noinline int g_mountver_ident_matches(struct g_geom *gp) { struct g_consumer *cp; -- 2.37.1 From 73085baed1cb671c595b36c97352ac37509cf70a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 28 Jan 2017 16:43:27 +0100 Subject: [PATCH 135/310] geom/mountver: Bump log level for G_MOUNTVER_LOGREQ to 3 to reduce noise when debugging tasting Obtained from: ElectroBSD --- sys/geom/mountver/g_mountver.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/geom/mountver/g_mountver.h b/sys/geom/mountver/g_mountver.h index 34ca831384f6..b95545c7f607 100644 --- a/sys/geom/mountver/g_mountver.h +++ b/sys/geom/mountver/g_mountver.h @@ -41,7 +41,7 @@ #define G_MOUNTVER_DEBUG(lvl, ...) \ _GEOM_DEBUG("GEOM_MOUNTVER", g_mountver_debug, (lvl), NULL, __VA_ARGS__) #define G_MOUNTVER_LOGREQ(bp, ...) \ - _GEOM_DEBUG("GEOM_MOUNTVER", g_mountver_debug, 2, (bp), __VA_ARGS__) + _GEOM_DEBUG("GEOM_MOUNTVER", g_mountver_debug, 3, (bp), __VA_ARGS__) struct g_mountver_softc { TAILQ_HEAD(, bio) sc_queue; -- 2.37.1 From 5545ffcfebd0542b180002d5c94c4b85063ca3e3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 29 Jan 2017 14:49:10 +0100 Subject: [PATCH 136/310] release/rc.local: Let the memstick create a script to self-reproduce Only works if there's enough free space at the end of the device the image is located on. Before trying to reproduce in a virtualized environment like bhyve increase the memstick size with something like: truncate -s +20G [...]/ElectroBSD-r314503-7314f38a400e.img Obtained from: ElectroBSD --- release/rc.local | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/release/rc.local b/release/rc.local index 7acf5070ef5e..776ab73e9719 100755 --- a/release/rc.local +++ b/release/rc.local @@ -10,6 +10,35 @@ MACHINE=`uname -m` +# create a script that can be used to reproduce the image with itself +# (provided there is enough space at the end to create another partition) +cat << 'EOF' > /tmp/reproduce-from-image.sh +set -x -e +# Fingers crossed that the first disk is the boot disk ... +DISK=$(geom disk status -s | head -n 1 | cut -f 1 -w) +JAILROOT="$(mktemp -d /tmp/ElectroBSD-jail-XXXXXX)" +SWAP_PARTITION_INDEX=3 +BUILD_PARTITION_INDEX=4 + +gpart recover "${DISK}" +gpart add -t freebsd-swap -s 1500M "${DISK}" +swapon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +dumpon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +gpart add -t freebsd-ufs "${DISK}" +# gpart partition changes made in bhyve are currently +# lost. The cause hasn't been diagnosed yet. +# Explicitly include the partition layout in the +# presumably logged output so it can be restored +# afterwards. +gpart show "${DISK}" # More convenient format for humans +gpart backup "${DISK}" # Same data in format understood by "gpart restore" +newfs "/dev/${DISK}p${BUILD_PARTITION_INDEX}" +mount -o async,noatime "/dev/${DISK}p${BUILD_PARTITION_INDEX}" "${JAILROOT}" +tar xf /usr/electrobsd-dist/base.txz -C "${JAILROOT}" +tar xf /usr/electrobsd-dist/src.txz -C "${JAILROOT}" +time jail -c path="${JAILROOT}" mount.devfs host.hostname=ElectroBSD command=sh /usr/src/reproduce.sh +EOF + # resolv.conf from DHCP ends up in here, so make sure the directory exists mkdir /tmp/bsdinstall_etc -- 2.37.1 From 5ae59df58819223d43f800aef4d75a59bea10166 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 25 Apr 2017 20:01:31 +0200 Subject: [PATCH 137/310] release/rc.local: Add /tmp/savecore-and-go-to-jail script Obtained from: ElectroBSD --- release/rc.local | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/release/rc.local b/release/rc.local index 776ab73e9719..a5977c3a8f84 100755 --- a/release/rc.local +++ b/release/rc.local @@ -39,6 +39,33 @@ tar xf /usr/electrobsd-dist/src.txz -C "${JAILROOT}" time jail -c path="${JAILROOT}" mount.devfs host.hostname=ElectroBSD command=sh /usr/src/reproduce.sh EOF +# create a script to recover a core dump and go to the jail to analyze it +cat << 'EOF' > /tmp/savecore-and-go-to-jail +set -x -e +# Fingers crossed that the first disk is the boot disk ... +DISK=$(geom disk status -s | head -n 1 | cut -f 1 -w) +JAILROOT="$(mktemp -d /tmp/ElectroBSD-jail-XXXXXX)" +SWAP_PARTITION_INDEX=3 +BUILD_PARTITION_INDEX=4 + +gpart recover "${DISK}" +gpart add -t freebsd-swap -s 1500M "${DISK}" +swapon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +dumpon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +gpart add -t freebsd-ufs "${DISK}" +# gpart partition changes made in bhyve are currently +# lost. The cause hasn't been diagnosed yet. +# Explicitly include the partition layout in the +# presumably logged output so it can be restored +# afterwards. +gpart show "${DISK}" # More convenient format for humans +gpart backup "${DISK}" # Same data in format understood by "gpart restore" +fsck -C -y "/dev/${DISK}p${BUILD_PARTITION_INDEX}" +mount -o async,noatime "/dev/${DISK}p${BUILD_PARTITION_INDEX}" "${JAILROOT}" +savecore -v "${JAILROOT}/var/crash" "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +time jail -c path="${JAILROOT}" mount.devfs host.hostname=ElectroBSD command=sh +EOF + # resolv.conf from DHCP ends up in here, so make sure the directory exists mkdir /tmp/bsdinstall_etc -- 2.37.1 From 7a791b9218d1896815e1271ffe44fb5f06006f8b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 30 Mar 2015 15:24:05 +0200 Subject: [PATCH 138/310] ggatel: Optionally retry in case of failed reads and writes ... after waiting five seconds and reopening the device. This allows to use ggatel to work around moderate USB flakiness which can result in device disconnects that are extremely annoying if ZFS on geli is involved. To prevent data loss if multiple device disappear at the same time and reappear with different names, the disk ident is checked to confirm that the device is the expected one. As a side-effect retrying will not work for file-backed memory disks (which currently have no disk ident). Obtained from: ElectroBSD --- sbin/ggate/ggatel/ggatel.c | 67 +++++++++++++++++++++++++++++++++++--- 1 file changed, 63 insertions(+), 4 deletions(-) diff --git a/sbin/ggate/ggatel/ggatel.c b/sbin/ggate/ggatel/ggatel.c index 4659cacbee17..88d0bf25ae46 100644 --- a/sbin/ggate/ggatel/ggatel.c +++ b/sbin/ggate/ggatel/ggatel.c @@ -55,6 +55,7 @@ static const char *path = NULL; static int unit = G_GATE_UNIT_AUTO; static unsigned flags = 0; static int force = 0; +static unsigned retries = 0; static unsigned sectorsize = 0; static unsigned timeout = G_GATE_TIMEOUT; @@ -62,10 +63,10 @@ static void usage(void) { - fprintf(stderr, "usage: %s create [-v] [-o ] " + fprintf(stderr, "usage: %s create [-v] [-o ] [-r ] " "[-s sectorsize] [-t timeout] [-u unit] \n", getprogname()); - fprintf(stderr, " %s rescue [-v] [-o ] <-u unit> " - "\n", getprogname()); + fprintf(stderr, " %s rescue [-v] [-o ] [-r ] " + "<-u unit> \n", getprogname()); fprintf(stderr, " %s destroy [-f] <-u unit>\n", getprogname()); fprintf(stderr, " %s list [-v] [-u unit]\n", getprogname()); exit(EXIT_FAILURE); @@ -87,6 +88,7 @@ g_gatel_serve(int fd) { struct g_gate_ctl_io ggio; size_t bsize; + char ident[DISK_IDENT_SIZE]; if (g_gate_verbose == 0) { if (daemon(0, 0) == -1) { @@ -99,8 +101,15 @@ g_gatel_serve(int fd) ggio.gctl_unit = unit; bsize = sectorsize; ggio.gctl_data = malloc(bsize); + + errno = 0; + if (retries && ioctl(fd, DIOCGIDENT, ident) != 0) { + g_gate_xlog("Failed to get disk ident for %s: %s", path, + strerror(errno)); + } for (;;) { int error; + int retries_left; once_again: ggio.gctl_length = bsize; ggio.gctl_error = 0; @@ -132,6 +141,8 @@ g_gatel_serve(int fd) strerror(error)); } + retries_left = retries; +retry_request: error = 0; switch (ggio.gctl_cmd) { case BIO_READ: @@ -147,6 +158,11 @@ g_gatel_serve(int fd) if (pread(fd, ggio.gctl_data, ggio.gctl_length, ggio.gctl_offset) == -1) { error = errno; + g_gate_log(LOG_ERR, "Failed to read %d" + " bytes at offset %d from %s: %s", + ggio.gctl_length, + (intmax_t)ggio.gctl_offset, path, + strerror(error)); } } break; @@ -155,12 +171,47 @@ g_gatel_serve(int fd) if (pwrite(fd, ggio.gctl_data, ggio.gctl_length, ggio.gctl_offset) == -1) { error = errno; + g_gate_log(LOG_ERR, "Failed to write %d bytes" + " at offset %jd to %s: %s", + ggio.gctl_length, + (intmax_t)ggio.gctl_offset, path, + strerror(error)); } break; default: error = EOPNOTSUPP; } + if (error && error != EOPNOTSUPP) { + if (retries_left > 0) { + char ident_new[DISK_IDENT_SIZE]; + + close(fd); + retries_left--; + sleep(5); + fd = open(path, g_gate_openflags(flags) | + O_DIRECT | O_FSYNC); + if (fd == -1) { + err(EXIT_FAILURE, "Cannot open %s", + path); + } + if (ioctl(fd, DIOCGIDENT, ident_new) != 0) { + g_gate_xlog("Failed to get disk ", + "ident for %s: %s", path, + strerror(errno)); + } + if (strcmp(ident, ident_new) != 0) { + g_gate_xlog("Disk ident for %s " + "changed from %s to %s. Reuse " + "could cause data loss.", path, + ident, ident_new); + } + g_gate_log(LOG_ERR, "Retrying after reopening " + "%s (%s). Retries left: %d", path, ident, + retries_left); + goto retry_request; + } + } ggio.gctl_error = error; g_gate_ioctl(G_GATE_CMD_DONE, &ggio); } @@ -232,7 +283,7 @@ main(int argc, char *argv[]) for (;;) { int ch; - ch = getopt(argc, argv, "fo:s:t:u:v"); + ch = getopt(argc, argv, "fo:r:s:t:u:v"); if (ch == -1) break; switch (ch) { @@ -255,6 +306,14 @@ main(int argc, char *argv[]) "Invalid argument for '-o' option."); } break; + case 'r': + if (action != CREATE && action != RESCUE) + usage(); + errno = 0; + retries = strtoul(optarg, NULL, 10); + if (retries == 0 && errno != 0) + errx(EXIT_FAILURE, "Invalid retry count."); + break; case 's': if (action != CREATE) usage(); -- 2.37.1 From 6890fd6aa9b4ce41bbb35e979331bfb6c8cedc17 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 26 Apr 2015 17:54:14 +0200 Subject: [PATCH 139/310] ggatel.8: Document the shiny new -r option Obtained from: ElectroBSD --- sbin/ggate/ggatel/ggatel.8 | 61 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/sbin/ggate/ggatel/ggatel.8 b/sbin/ggate/ggatel/ggatel.8 index 7facee0c5369..d80a27eb54a4 100644 --- a/sbin/ggate/ggatel/ggatel.8 +++ b/sbin/ggate/ggatel/ggatel.8 @@ -35,6 +35,7 @@ .Cm create .Op Fl v .Op Fl o Cm ro | wo | rw +.Op Fl r Ar retries .Op Fl s Ar sectorsize .Op Fl t Ar timeout .Op Fl u Ar unit @@ -51,6 +52,7 @@ .Cm rescue .Op Fl v .Op Fl o Cm ro | wo | rw +.Op Fl r Ar retries .Fl u Ar unit .Ar path .Sh DESCRIPTION @@ -103,6 +105,19 @@ or read-write .Pq Cm rw . Default is .Cm rw . +.It Fl r Ar retries +Number of times a failed request should be retried before forwarding +the error to the kernel. +Between retries, +.Nm ggatel +waits for five seconds and reopens the device in case it temporarily +disappeared. +The reopened device is only used if the disk identification did not +change. +This option is useful when using unreliable USB devices as geli +consumer (as long as the device loss does not cause the USB +stack to deadlock). +By default failed requests are not retried. .It Fl s Ar sectorsize Sector size for .Nm ggate @@ -145,6 +160,52 @@ umount /secret gbde detach ggate5 ggatel destroy -u 5 .Ed + +Scrub a pool on an USB device that occasionally disappears: +.Bd -literal -offset indent +$ glabel list da0 +Geom name: da0 +Providers: +1. Name: label/extreme + Mediasize: 4023385600 (3.7G) + Sectorsize: 512 + Mode: r0w0e0 + secoffset: 0 + offset: 0 + seclength: 7858175 + length: 4023385600 + index: 0 +Consumers: +1. Name: da0 + Mediasize: 4023386112 (3.7G) + Sectorsize: 512 + Mode: r0w0e0 + +$ sudo ggatel create -r 2 /dev/da0 +ggate0 +$ glabel list da0 +glabel: No such geom: da0. +glabel list ggate0 +Geom name: ggate0 +Providers: +1. Name: label/extreme + Mediasize: 4023385600 (3.7G) + Sectorsize: 512 + Mode: r0w0e0 + secoffset: 0 + offset: 0 + seclength: 7858175 + length: 4023385600 + index: 0 +Consumers: +1. Name: ggate0 + Mediasize: 4023386112 (3.7G) + Sectorsize: 512 + Mode: r0w0e0 + +$ zogftw import extreme +$ sudo zpool scrub extreme +.Ed .Sh SEE ALSO .Xr geom 4 , .Xr gbde 8 , -- 2.37.1 From 94b8fc7fce98d0881cd085dfbe348059a03cf0f6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 26 Mar 2017 18:42:38 +0200 Subject: [PATCH 140/310] ELECTRO_BLOAT: Disable GEOM_RAID which can cause deadlocks ... even if it's not intentionally being used: 13 100020 geom g_event mi_switch+0xd2 sleepq_wait+0x3a _sleep+0x257 biowait+0x90 g_read_data+0x7e g_raid_md_taste_sii+0xde g_raid_taste+0x16b g_new_provider_event+0xca g_run_events+0x186 fork_exit+0x85 fork_trampoline+0xe 31379 100863 zpool - mi_switch+0xd2 sleepq_timedwait+0x42 _sleep+0x237 g_waitidle+0xa1 userret+0x55 amd64_syscall+0x52f Xfast_syscall+0xfb 60137 101433 ggatel - mi_switch+0xd2 sleepq_timedwait+0x42 _sleep+0x237 g_waitidle+0xa1 userret+0x55 amd64_syscall+0x52f Xfast_syscall+0xfb Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index 3bd60d2df610..f440d4222d02 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -45,7 +45,11 @@ options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. -options GEOM_RAID # Soft RAID functionality. +# This is unlikely to be intentionally used be ElectroBSD +# users but its mere presence in the kernel can result in +# deadlocks while tasting flaky providers and the deadlock +# affects other geom classes as well. +nooptions GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization options COMPAT_FREEBSD32 # Compatible with i386 binaries options COMPAT_FREEBSD10 # Compatible with FreeBSD10 -- 2.37.1 From 4bf7123cb327a4bdeb6d1f10fe817c75fcc96d24 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 1 Apr 2017 17:35:26 +0200 Subject: [PATCH 141/310] release/scripts/image-checksum.sh: Add a bunch of additional mtree keywords to see if they make a difference Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 30ac62a6c168..18e3a521880f 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -49,7 +49,7 @@ UFS_PARTITION=p2 EXPECTED_PARTITIONS=2 MOUNTPOINT=/mnt VERBOSE=0 -MTREE_KEYWORDS=size,time,uid,gid,sha256 +MTREE_KEYWORDS=size,time,mode,uid,gid,sha256,device,flags,link,nlink,type REUSE_EXISTING_CACHE_FILES=false verbose_log() { -- 2.37.1 From 652485604ed1655aecea0d8fee7078c99261d1db Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 7 Apr 2017 09:33:34 +0200 Subject: [PATCH 142/310] Let image-checksum check inodes using ls Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 32 +++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 18e3a521880f..17f062200b61 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -76,6 +76,22 @@ create_mtree_spec_file() { umount "${MOUNTPOINT}" || return 1 } +create_ls_file() { + local md_unit ls_file + + md_unit="${1}" + ls_file="${2}" + + verbose_log "Mounting /dev/md${md_unit}${UFS_PARTITION} at ${MOUNTPOINT}" + mount -o ro "/dev/md${md_unit}${UFS_PARTITION}" "${MOUNTPOINT}" || return 1 + + verbose_log "Running ls to get inodes, saving output in ${ls_file}" + env -i LC_ALL=C ls -liR -D "%Y-%m-%d %H:%M:%S" "${MOUNTPOINT}" > "${ls_file}" || return 1 + + verbose_log "Unmounting ${MOUNTPOINT} ..." + umount "${MOUNTPOINT}" || return 1 +} + partition_count_acceptable() { local md_unit="${1}" @@ -89,7 +105,9 @@ partition_count_acceptable() { generate_partial_image_checksum() { local image_file \ - md_unit spec_file gpart_file + md_unit spec_file gpart_file ls_file \ + gpart_checksum bootcode_checksum ls_checksum \ + weak_image_checksum mtree_checksum image_file="${1}" spec_file="${image_file}.mtree" @@ -102,6 +120,11 @@ generate_partial_image_checksum() { echo "gpart file ${gpart_file} already exists" ${REUSE_EXISTING_CACHE_FILES} || return 1 fi + ls_file="${image_file}.ls" + if [ -f "${ls_file}" ]; then + echo "ls file ${ls_file} already exists" + ${REUSE_EXISTING_CACHE_FILES} || return 1 + fi md_unit=$(mdconfig -o readonly -n -f "${image_file}") if [ $? != 0 ]; then @@ -116,6 +139,9 @@ generate_partial_image_checksum() { if [ ! -f "${gpart_file}" ]; then gpart list "md${md_unit}" | sed -E -e "s@(: md)${md_unit}@\1X@" > "${gpart_file}" fi + if [ ! -f "${ls_file}" ]; then + create_ls_file "${md_unit}" "${ls_file}" || return 1 + fi gpart_checksum=$(sha256 -q "${gpart_file}") verbose_log "gpart checksum: ${gpart_checksum}" @@ -126,8 +152,10 @@ generate_partial_image_checksum() { verbose_log "Boot code checksum: ${bootcode_checksum}" mtree_checksum=$(sha256 -q "${spec_file}") verbose_log "mtree checksum: ${mtree_checksum}" + ls_checksum=$(sha256 -q "${ls_file}") + verbose_log "ls checksum: ${ls_checksum}" - weak_image_checksum=$(echo "${gpart_checksum} ${bootcode_checksum} ${mtree_checksum}" | sha256) + weak_image_checksum=$(echo "${gpart_checksum} ${bootcode_checksum} ${mtree_checksum} ${ls_checksum}" | sha256) echo "Partial image checksum for ${image_file}: ${weak_image_checksum}" } -- 2.37.1 From f509663762f29fce0d2b7374bbdfdb06b3188af4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 4 Apr 2017 14:29:31 +0200 Subject: [PATCH 143/310] release/scripts/strip-freebsd.sh: Ditch qlnx ... which contains source-less microcode in sys/dev/qlnx/qlnxe/ecore_init_values.h. It was added in r316485/c52917fd11 but not MFC'd yet. This commit should make sure that the MFC will break the ElectroBSD built and can be fixed if the upstream report is ignored. Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index e805f9cab8e4..c6875e6baab8 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -86,7 +86,7 @@ get_tainted_sys_contrib_devs() { # and may cause build failures without it. get_tainted_sys_devs() { # bce has already been taken care of by get_bce_files() above - echo "bxe ctau cx cxgb cxgbe ispfw qlxgbe" \ + echo "bxe ctau cx cxgb cxgbe ispfw qlxgbe qlnx" \ "it tw" } -- 2.37.1 From d6b3ed0a37f65a25fe220fdfcbe0ef94c28a58e4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 7 Apr 2017 15:46:05 +0200 Subject: [PATCH 144/310] release/Makefile: Try to debug /etc/rc.local mode issue and fix it Obtained from: ElectroBSD --- release/Makefile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/release/Makefile b/release/Makefile index a315798ec66d..d35e00bc6f22 100644 --- a/release/Makefile +++ b/release/Makefile @@ -213,6 +213,10 @@ disc1: packagesystem echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf cp ${.CURDIR}/rc.local ${.TARGET}/etc + umask + ls -l ${.TARGET}/etc/rc.local + chmod 550 ${.TARGET}/etc/rc.local + ls -l ${.TARGET}/etc/rc.local touch ${.TARGET} bootonly: packagesystem -- 2.37.1 From 28c7d410418617eb714f11af8479f2319e8d73a5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 8 Apr 2017 15:41:03 +0200 Subject: [PATCH 145/310] usr.sbin/cdcontrol: Retry closing after ENOTTY failures (XXX: not really) This doesn't actually work but the commit is kept around as a reminder to fix it. Obtained from: ElectroBSD --- usr.sbin/cdcontrol/cdcontrol.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/usr.sbin/cdcontrol/cdcontrol.c b/usr.sbin/cdcontrol/cdcontrol.c index 546c20b6f94a..0fa1f7a02534 100644 --- a/usr.sbin/cdcontrol/cdcontrol.c +++ b/usr.sbin/cdcontrol/cdcontrol.c @@ -389,6 +389,10 @@ run(int cmd, char *arg) (void) ioctl (fd, CDIOCALLOW); rc = ioctl (fd, CDIOCCLOSE); + if (rc == ENOTTY) { + warnx("Retrying"); + rc = ioctl (fd, CDIOCCLOSE); + } if (rc < 0) return (rc); close(fd); -- 2.37.1 From 16f39641f2caa1bed7d7142e746aae92c6d86c3a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 16 Apr 2017 13:35:11 +0200 Subject: [PATCH 146/310] Add ELECTRO_BEER_DEBUG kernel which is ELECTRO_BEER plus debug options Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BEER_DEBUG | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BEER_DEBUG diff --git a/sys/amd64/conf/ELECTRO_BEER_DEBUG b/sys/amd64/conf/ELECTRO_BEER_DEBUG new file mode 100644 index 000000000000..22c1b306669a --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BEER_DEBUG @@ -0,0 +1,8 @@ +include ELECTRO_BEER + +ident ELECTRO_BEER_DEBUG + +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed -- 2.37.1 From fce4cecc64bceda46b56288cd635c7e890759227 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 20 Aug 2015 10:12:27 +0200 Subject: [PATCH 147/310] pw(8): Increase minimal random password length to 32 It's not obvious to me why the length is randomized as well in the first place but whatever ... Obtained from: ElectroBSD --- usr.sbin/pw/pw_user.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/usr.sbin/pw/pw_user.c b/usr.sbin/pw/pw_user.c index 2eec317b5e5b..552ce78a6665 100644 --- a/usr.sbin/pw/pw_user.c +++ b/usr.sbin/pw/pw_user.c @@ -514,13 +514,13 @@ static char * pw_password(struct userconf * cnf, char const * user, bool dryrun) { int i, l; - char pwbuf[32]; + char pwbuf[41]; switch (cnf->default_password) { case P_NONE: /* No password at all! */ return ""; case P_RANDOM: /* Random password */ - l = (arc4random() % 8 + 8); /* 8 - 16 chars */ + l = sizeof(pwbuf) - 1 - (arc4random() % 8); /* 32 - 40 chars */ for (i = 0; i < l; i++) pwbuf[i] = chars[arc4random_uniform(sizeof(chars)-1)]; pwbuf[i] = '\0'; -- 2.37.1 From 5bf951781bd9d6e91020d36e9ad6b8767b5521df Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 14 May 2016 16:53:50 +0200 Subject: [PATCH 148/310] uma.h: Rename uma_prealloc()'s second argument to nitems to increase consistency PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209716 PR submission date: 2016-05-23 Obtained from: ElectroBSD --- sys/vm/uma.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/vm/uma.h b/sys/vm/uma.h index dc8bd07d299e..f09bdfb07a1f 100644 --- a/sys/vm/uma.h +++ b/sys/vm/uma.h @@ -650,14 +650,14 @@ smr_t uma_zone_get_smr(uma_zone_t zone); * * Arguments: * zone The zone to fill - * itemcnt The number of items to reserve + * nitems The number of items to reserve * * Returns: * Nothing * * NOTE: This is blocking and should only be done at startup */ -void uma_prealloc(uma_zone_t zone, int itemcnt); +void uma_prealloc(uma_zone_t zone, int nitems); /* * Used to determine if a fixed-size zone is exhausted. -- 2.37.1 From a7ae045f9bc6d1b11ab0be548c4a6de14ec79647 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 16 May 2016 12:28:50 +0200 Subject: [PATCH 149/310] uma.h: Stop claiming that uma_zcreate() may return NULL ... 'if the wait flag is not set'. The function does not actually accept a "wait flag". Internally it unconditionally uses the M_WAITOK flag when calling zone_alloc_item(). Obtained from: ElectroBSD --- sys/vm/uma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/vm/uma.h b/sys/vm/uma.h index f09bdfb07a1f..5bec472c611d 100644 --- a/sys/vm/uma.h +++ b/sys/vm/uma.h @@ -174,7 +174,7 @@ typedef void (*uma_release)(void *arg, void **store, int count); * * Returns: * A pointer to a structure which is intended to be opaque to users of - * the interface. The value may be null if the wait flag is not set. + * the interface. */ uma_zone_t uma_zcreate(const char *name, size_t size, uma_ctor ctor, uma_dtor dtor, uma_init uminit, uma_fini fini, -- 2.37.1 From e2332d626a03e29cea7e44c6f669e728bc25cd91 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 24 Apr 2017 21:45:28 +0200 Subject: [PATCH 150/310] Add ELECTRO_BLOAT_DEBUG which is ELECTRO_BLOAT plus debug settings Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT_DEBUG | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BLOAT_DEBUG diff --git a/sys/amd64/conf/ELECTRO_BLOAT_DEBUG b/sys/amd64/conf/ELECTRO_BLOAT_DEBUG new file mode 100644 index 000000000000..2d06dcf4e2d8 --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BLOAT_DEBUG @@ -0,0 +1,9 @@ +include ELECTRO_BLOAT + +ident ELECTRO_BLOAT_DEBUG + +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +options DEBUG_MEMGUARD -- 2.37.1 From 5f796c6a99e1c5c249e95f49b57d8da3341ca8b6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 17 Mar 2016 12:51:02 +0100 Subject: [PATCH 151/310] sys/cam: Retry in case of 'uncorrectable' errors At least in case of the LITE-ON DVDRW SOHW-1693S with firmware KC4B, they are recoverable. Maybe this should be done as device-specific quirk. Obtained from: ElectroBSD --- sys/cam/scsi/scsi_all.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/cam/scsi/scsi_all.c b/sys/cam/scsi/scsi_all.c index 6c56d4e3b234..044aa76c5468 100644 --- a/sys/cam/scsi/scsi_all.c +++ b/sys/cam/scsi/scsi_all.c @@ -1418,8 +1418,8 @@ static struct asc_table_entry asc_table[] = { { SST(0x11, 0x04, SS_FATAL|EIO, "Unrecovered read error - auto reallocate failed") }, /* WRO B */ - { SST(0x11, 0x05, SS_FATAL|EIO, - "L-EC uncorrectable error") }, + { SST(0x11, 0x05, SS_RDEF, + "L-EC 'uncorrectable' error") }, /* WRO B */ { SST(0x11, 0x06, SS_FATAL|EIO, "CIRC unrecovered error") }, -- 2.37.1 From db6f8b287cf9db745fc02bc018b592339c9fc9b1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 27 Apr 2017 19:45:55 +0200 Subject: [PATCH 152/310] kern_racct: Add sysctl to modify the delay between resource limit checks etc. Obtained from: ElectroBSD --- sys/kern/kern_racct.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/sys/kern/kern_racct.c b/sys/kern/kern_racct.c index 94624e88af51..dc77be97d22e 100644 --- a/sys/kern/kern_racct.c +++ b/sys/kern/kern_racct.c @@ -84,6 +84,11 @@ SYSCTL_BOOL(_kern_racct, OID_AUTO, enable, CTLFLAG_RDTUN, &racct_enable, SYSCTL_UINT(_kern_racct, OID_AUTO, pcpu_threshold, CTLFLAG_RW, &pcpu_threshold, 0, "Processes with higher %cpu usage than this value can be throttled."); +static unsigned int racctd_delay = 0; +SYSCTL_UINT(_kern_racct, OID_AUTO, racctd_delay, CTLFLAG_RWTUN, + &racctd_delay, 0, "Number of ticks to pause between resource " + "usage/limit updates."); + /* * How many seconds it takes to use the scheduler %cpu calculations. When a * process starts, we compute its %cpu usage by dividing its runtime by the @@ -1252,6 +1257,9 @@ racctd(void) ASSERT_RACCT_ENABLED(); + if (racctd_delay == 0) + racctd_delay = hz; + for (;;) { racct_decay(); @@ -1327,7 +1335,7 @@ racctd(void) PROC_UNLOCK(p); } sx_sunlock(&allproc_lock); - pause("-", hz); + pause("-", racctd_delay); } } -- 2.37.1 From 77c8c9a51de6675e4dce2668e16533d54c0b67a2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 19 May 2017 11:53:57 +0200 Subject: [PATCH 153/310] sys/net: Disable MAC address caching by default MAC address caching was added upstream in r318160/0cfdb3c3056d9e and allows to retrieve the orginal MAC address with ifconfig, even if it has been previously overwritten. This currently can be done from jails and makes system fingerprinting easier which may be undesirable. Now it has to be explicitly enabled by setting: net.link.cache_mac_addresses=1 before the NIC is initialized. Note that jails can still access the (potentially randomized) MAC address the NICs are currently using. The "protection" offered by this commit is therefore quite limitted. Obtained from: ElectroBSD --- sys/net/if.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/sys/net/if.c b/sys/net/if.c index b8aadbf03041..1c83a30318fd 100644 --- a/sys/net/if.c +++ b/sys/net/if.c @@ -221,6 +221,13 @@ SYSCTL_UINT(_net, OID_AUTO, ifdescr_maxlen, CTLFLAG_RW, &ifdescr_maxlen, 0, "administrative maximum length for interface description"); +/* Try to cache the original MAC addresses */ +static int cache_mac_addresses = 0; +SYSCTL_INT(_net_link, OID_AUTO, cache_mac_addresses, CTLFLAG_RWTUN, + &cache_mac_addresses, 0, + "Try to cache original MAC addresses allowing " + "ifconfig to display them after being overwritten."); + static MALLOC_DEFINE(M_IFDESCR, "ifdescr", "ifnet descriptions"); /* global sx for non-critical path ifdescr */ @@ -945,7 +952,7 @@ if_attach_internal(struct ifnet *ifp, int vmove, struct if_clone *ifc) /* Reliably crash if used uninitialized. */ ifp->if_broadcastaddr = NULL; - if (ifp->if_type == IFT_ETHER) { + if (cache_mac_addresses != 0 && ifp->if_type == IFT_ETHER) { ifp->if_hw_addr = malloc(ifp->if_addrlen, M_IFADDR, M_WAITOK | M_ZERO); } -- 2.37.1 From c1a5f41dbdf439a6d4e2eb05d1ed1fa3e5c8e7d9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 21 May 2017 14:27:06 +0200 Subject: [PATCH 154/310] cddl/lib/libdtrace: Remove -O2 for now to get better core dumps Obtained from: ElectroBSD --- cddl/lib/libdtrace/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/cddl/lib/libdtrace/Makefile b/cddl/lib/libdtrace/Makefile index 05b183215f4c..2e44ef59eeff 100644 --- a/cddl/lib/libdtrace/Makefile +++ b/cddl/lib/libdtrace/Makefile @@ -90,6 +90,7 @@ CFLAGS+= -I${.OBJDIR} -I${.CURDIR} \ -I${OPENSOLARIS_USR_DISTDIR}/lib/libctf/common \ -I${OPENSOLARIS_USR_DISTDIR}/lib/libdtrace/common \ -I${OPENSOLARIS_SYS_DISTDIR}/uts/common +CFLAGS:= ${CFLAGS:S/-O2//} #CFLAGS+= -DYYDEBUG -- 2.37.1 From 843c1ad7bbb97e4d70d21fe2534be358b712041b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 1 Jun 2017 16:45:03 +0200 Subject: [PATCH 155/310] Don't try to build cxgbetool (XXX: Improve commit message) It's not useful without cxgbe which isn't shipped with ElectroBSD and relies on t4_ioctl.h which isn't shipped either. Also don't try to build mlx5tool. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 16d6b62c74b8..e6c7545fa762 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -359,11 +359,9 @@ __DEFAULT_NO_OPTIONS+=OPENSSL_KTLS # profiling won't work on MIPS64 because there is only assembly for o32 BROKEN_OPTIONS+=PROFILE .endif -.if ${__T} != "aarch64" && ${__T} != "amd64" && ${__T} != "i386" && \ - ${__T} != "powerpc64" -BROKEN_OPTIONS+=CXGBETOOL -BROKEN_OPTIONS+=MLX5TOOL -.endif + +__DEFAULT_NO_OPTIONS+=CXGBETOOL +__DEFAULT_NO_OPTIONS+=MLX5TOOL # HyperV is currently x86-only .if ${__T} != "amd64" && ${__T} != "i386" -- 2.37.1 From 962d044aef5d7e2bda36ea8e682a5a640191bbdb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 24 Jun 2017 12:04:15 +0200 Subject: [PATCH 156/310] Detach upgt from the build It relies on proprietary firmware (not part of the upstream tree). Obtained from: ElectroBSD --- sys/modules/usb/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/modules/usb/Makefile b/sys/modules/usb/Makefile index 1a34b221efa0..a113661e289c 100644 --- a/sys/modules/usb/Makefile +++ b/sys/modules/usb/Makefile @@ -46,7 +46,7 @@ MAKE+=" DEBUG_FLAGS+=-DUSB_REQ_DEBUG" SUBDIR = usb SUBDIR += ${_dwc_otg} ehci ${_musb} ohci uhci xhci ${_uss820dci} \ ${_atmegadci} ${_avr32dci} ${_rsu} ${_rsufw} ${_saf1761otg} -SUBDIR += ${_rum} ${_run} ${_runfw} ${_uath} upgt usie ural ${_zyd} ${_urtw} +SUBDIR += ${_rum} ${_run} ${_runfw} ${_uath} usie ural ${_zyd} ${_urtw} SUBDIR += atp cfumass uhid uhid_snes ukbd ums udbp uep wmt wsp ugold uled \ usbhid SUBDIR += ucom u3g uark ubsa ubser uchcom ucycom ufoma uftdi ugensa uipaq ulpt \ -- 2.37.1 From 57202e61f6e2fcd933d78ea47e09f0552112e636 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 10 Jul 2017 14:37:11 +0200 Subject: [PATCH 157/310] sys/kern: Follow OpenBSD's lead and remove TIOCSTI support ... to prevent tty hijacking issues like CVE-2005-4890. TIOCSTI is still used by by mail, but this could (probably) be fixed by adding a "#undef TIOCSTI" or removing the definition from sys/sys/ttycom.h. Additionally TIOCSTI is still used in tcsh, but as tcsh isn't compiled on ElectroBSD we don't care. Obtained from: ElectroBSD --- sys/kern/tty.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/sys/kern/tty.c b/sys/kern/tty.c index 8dfe5e93780f..3308be22e568 100644 --- a/sys/kern/tty.c +++ b/sys/kern/tty.c @@ -585,7 +585,6 @@ ttydev_ioctl(struct cdev *dev, u_long cmd, caddr_t data, int fflag, case TIOCSPGRP: case TIOCSTART: case TIOCSTAT: - case TIOCSTI: case TIOCSTOP: case TIOCSWINSZ: #if 0 @@ -1952,14 +1951,7 @@ tty_generic_ioctl(struct tty *tp, u_long cmd, void *data, int fflag, tty_info(tp); return (0); case TIOCSTI: - if ((fflag & FREAD) == 0 && priv_check(td, PRIV_TTY_STI)) - return (EPERM); - if (!tty_is_ctty(tp, td->td_proc) && - priv_check(td, PRIV_TTY_STI)) - return (EACCES); - ttydisc_rint(tp, *(char *)data, 0); - ttydisc_rint_done(tp); - return (0); + return (EIO); } #ifdef COMPAT_43TTY -- 2.37.1 From dbcae2b961994ab49f21d3ace0a900b19fc554c1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 28 Jul 2017 12:07:47 +0200 Subject: [PATCH 158/310] cardbus: Do not give the world read and write permission to the cardbus device Obtained from: ElectroBSD --- sys/dev/cardbus/cardbus_device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/cardbus/cardbus_device.c b/sys/dev/cardbus/cardbus_device.c index 0de25572fa9c..e8513c598348 100644 --- a/sys/dev/cardbus/cardbus_device.c +++ b/sys/dev/cardbus/cardbus_device.c @@ -116,7 +116,7 @@ cardbus_device_create(struct cardbus_softc *sc, struct cardbus_devinfo *devi, cardbus_device_buffer_cis(parent, child, &devi->sc_cis); minor = (device_get_unit(sc->sc_dev) << 8) + devi->pci.cfg.func; unit = device_get_unit(sc->sc_dev); - devi->sc_cisdev = make_dev(&cardbus_cdevsw, minor, 0, 0, 0666, + devi->sc_cisdev = make_dev(&cardbus_cdevsw, minor, 0, 0, 0660, "cardbus%d.%d.cis", unit, devi->pci.cfg.func); if (devi->pci.cfg.func == 0) make_dev_alias(devi->sc_cisdev, "cardbus%d.cis", unit); -- 2.37.1 From f296508b49c55dcd235ce188eb17b522f60283a5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 4 Aug 2017 12:06:37 +0200 Subject: [PATCH 159/310] kern_fcntl(): Unconditionally init tmp Obtained from: ElectroBSD --- sys/kern/kern_descrip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/kern_descrip.c b/sys/kern/kern_descrip.c index ff999cc82f97..90e01fbdaf1c 100644 --- a/sys/kern/kern_descrip.c +++ b/sys/kern/kern_descrip.c @@ -487,7 +487,7 @@ kern_fcntl(struct thread *td, int fd, int cmd, intptr_t arg) uint64_t bsize; off_t foffset; - error = 0; + tmp = error = 0; flg = F_POSIX; p = td->td_proc; fdp = p->p_fd; -- 2.37.1 From e5923ee4b94786454f4792c5740d1072e7afb4f0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 6 Aug 2017 13:32:21 +0200 Subject: [PATCH 160/310] sys/kern/sched_ule.c: Don't add an empty line to the generated spec Obtained from: ElectroBSD --- sys/kern/sched_ule.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c index 4cde748ea9b7..161a6f197cac 100644 --- a/sys/kern/sched_ule.c +++ b/sys/kern/sched_ule.c @@ -3194,7 +3194,7 @@ sysctl_kern_sched_topology_spec(SYSCTL_HANDLER_ARGS) sbuf_printf(topo, "\n"); err = sysctl_kern_sched_topology_spec_internal(topo, cpu_top, 1); - sbuf_printf(topo, "\n"); + sbuf_printf(topo, ""); if (err == 0) { err = sbuf_finish(topo); -- 2.37.1 From 4a36845a15255a2000ecbfa07e1ec2bd7d3bc711 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 18 Jul 2017 21:33:50 +0200 Subject: [PATCH 161/310] sys/amd64/conf/ELECTRO_BLOAT: Add options CAM_IOSCHED_DYNAMIC Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index f440d4222d02..ae26c7af8e03 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -16,6 +16,8 @@ options ACCEPT_FILTER_HTTP # works around various bugs that only affect the module build. device pf +options CAM_IOSCHED_DYNAMIC + ############################################################################## # Everything below comes from GENERIC, but "offending" lines have been removed ############################################################################## -- 2.37.1 From 7ab1ccc6d88aa350dda4c6c4a0c8142e0c26b092 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 10 Aug 2017 14:20:25 +0200 Subject: [PATCH 162/310] geom_nop(4): Include BIO_ZONE commands in the statistics Obtained from: ElectroBSD --- sys/geom/nop/g_nop.c | 6 ++++++ sys/geom/nop/g_nop.h | 1 + 2 files changed, 7 insertions(+) diff --git a/sys/geom/nop/g_nop.c b/sys/geom/nop/g_nop.c index 57208a0744db..448c327d366f 100644 --- a/sys/geom/nop/g_nop.c +++ b/sys/geom/nop/g_nop.c @@ -266,6 +266,9 @@ g_nop_start(struct bio *bp) case BIO_CMD2: sc->sc_cmd2s++; break; + case BIO_ZONE: + sc->sc_zone_cmds++; + break; } mtx_unlock(&sc->sc_lock); @@ -445,6 +448,7 @@ g_nop_create(struct gctl_req *req, struct g_class *mp, struct g_provider *pp, sc->sc_cmd0s = 0; sc->sc_cmd1s = 0; sc->sc_cmd2s = 0; + sc->sc_zone_cmds = 0; sc->sc_readbytes = 0; sc->sc_wrotebytes = 0; TAILQ_INIT(&sc->sc_head_delay); @@ -896,6 +900,7 @@ g_nop_ctl_reset(struct gctl_req *req, struct g_class *mp) sc->sc_cmd0s = 0; sc->sc_cmd1s = 0; sc->sc_cmd2s = 0; + sc->sc_zone_cmds = 0; sc->sc_readbytes = 0; sc->sc_wrotebytes = 0; } @@ -967,6 +972,7 @@ g_nop_dumpconf(struct sbuf *sb, const char *indent, struct g_geom *gp, sbuf_printf(sb, "%s%ju\n", indent, sc->sc_cmd0s); sbuf_printf(sb, "%s%ju\n", indent, sc->sc_cmd1s); sbuf_printf(sb, "%s%ju\n", indent, sc->sc_cmd2s); + sbuf_printf(sb, "%s%ju\n", indent, sc->sc_zone_cmds); sbuf_printf(sb, "%s%ju\n", indent, sc->sc_readbytes); sbuf_printf(sb, "%s%ju\n", indent, diff --git a/sys/geom/nop/g_nop.h b/sys/geom/nop/g_nop.h index b8f086c56781..b0c149c348e5 100644 --- a/sys/geom/nop/g_nop.h +++ b/sys/geom/nop/g_nop.h @@ -72,6 +72,7 @@ struct g_nop_softc { uintmax_t sc_cmd1s; uintmax_t sc_cmd2s; uintmax_t sc_speedups; + uintmax_t sc_zone_cmds; uintmax_t sc_readbytes; uintmax_t sc_wrotebytes; char *sc_physpath; -- 2.37.1 From 2aa6f75e79c1a426a2a67207016699dc30e4ca4b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 11 Aug 2017 16:40:28 +0200 Subject: [PATCH 163/310] strip-freebsd.sh: Delete sendmail rc scripts as we don't ship the license either Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index c6875e6baab8..e5287317b07a 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -140,6 +140,14 @@ get_directories_to_ditch() { echo "${potential_directory}" fi done + + # These files aren't useful without sendmail itself. + # More importantly, some of them are licensed under the + # "Sendmail License" which isn't part of the directory. + # + # Deleting them prevents them from ending up in + # the source tarball without license text. + echo etc/sendmail } purify_cwd() { -- 2.37.1 From 2c446b59ce03e0d2fa06c3648ea48e5258f8f97a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 11 Aug 2017 17:41:16 +0200 Subject: [PATCH 164/310] release/scripts/strip-freebsd.sh: Ditch share/doc Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index e5287317b07a..d74f5b4c2d24 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -148,6 +148,10 @@ get_directories_to_ditch() { # Deleting them prevents them from ending up in # the source tarball without license text. echo etc/sendmail + + # Remove the "4.4BSD legacy docs". The content isn't + # needed and the license terms aren't obvious. + echo share/doc } purify_cwd() { -- 2.37.1 From 2b0f11d49f11f5483eaa418419996b28385423a0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 11 Aug 2017 20:53:54 +0200 Subject: [PATCH 165/310] share/mk/src.opts.mk: Disable SHAREDOCS to unbreak the build src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index e6c7545fa762..ce9792f95d83 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -155,7 +155,6 @@ __DEFAULT_YES_OPTIONS = \ SERVICESDB \ SETUID_LOGIN \ SHARED_TOOLCHAIN \ - SHAREDOCS \ STATS \ SYSCONS \ SYSTEM_COMPILER \ @@ -197,6 +196,7 @@ __DEFAULT_NO_OPTIONS = \ SORT_THREADS \ SVN \ SVNLITE \ + SHAREDOCS \ ZONEINFO_LEAPSECONDS_SUPPORT \ # LEFT/RIGHT. Left options which default to "yes" unless their corresponding @@ -255,6 +255,7 @@ __DEFAULT_NO_OPTIONS += \ ROUTED \ SENDMAIL \ SVNLITE \ + SHAREDOCS \ SOURCELESS \ SOURCELESS_HOST \ SOURCELESS_UCODE \ -- 2.37.1 From 724d8e128da6449d5fc3209d32674c974e50e4b0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 21 Aug 2017 18:56:06 +0200 Subject: [PATCH 166/310] share/mk: Disable AMD by default It's obsolete and already deprecated by upstream in favour of autofs src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index ce9792f95d83..b980bfaf4a80 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -232,6 +232,7 @@ __DEFAULT_YES_OPTIONS += \ # Disable a bunch of additional options that default to yes in FreeBSD __DEFAULT_NO_OPTIONS += \ ATM \ + AMD \ BLUETOOTH \ BSDINSTALL \ CCD \ -- 2.37.1 From 512f21c9c3cb357f005c374afbe28ab6a2e962a7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 12:05:18 +0200 Subject: [PATCH 167/310] share/mk/src.opts.mk: Disable NETGRAPH by default src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index b980bfaf4a80..19f2e46b36b7 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -136,7 +136,6 @@ __DEFAULT_YES_OPTIONS = \ MALLOC_PRODUCTION \ MANDOCDB \ NETCAT \ - NETGRAPH \ NLS_CATALOGS \ NS_CACHING \ NTP \ @@ -246,6 +245,7 @@ __DEFAULT_NO_OPTIONS += \ IPFILTER \ ISCSI \ LIB32 \ + NETGRAPH \ NDIS \ RBOOTD \ PC_SYSINSTALL \ -- 2.37.1 From c95c827d2b4886113f628fe5c7dca5c77da4b119 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 21 Aug 2017 19:05:13 +0200 Subject: [PATCH 168/310] strip-freebsd.sh: Add amd to the list of contrib directories to ditch Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index d74f5b4c2d24..75b48141a6d9 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -94,7 +94,7 @@ get_unused_contrib_dirs() { # XXX: gcc can't be deleted because parts of it are apparently # required to build libc. This should be investigated more thoroughly, # hopefully it can be fixed. - echo "apr apr-util ipfilter ofed sendmail serf subversion tcsh" + echo "amd apr apr-util ipfilter ofed sendmail serf subversion tcsh" } # Only includes directory that aren't architecture-specific -- 2.37.1 From a8b1a43f923f896401b47e4d908092683b2d5c0e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 21 Aug 2017 19:18:58 +0200 Subject: [PATCH 169/310] strip-freebsd.sh: Try to remove directories that reference unused contrib code Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 75b48141a6d9..b8be6b84ec9d 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -134,13 +134,24 @@ get_directories_to_ditch() { echo "sys/dev/${sys_dev}" done + # For each known unused directory remove it from + # "contrib" (where most of the code is located) + # and "usr.sbin" and "usr.bin" if a directory with + # the same name exists. for contrib_dir in $(get_unused_contrib_dirs); do - potential_directory="contrib/${contrib_dir}" + for potential_parent in contrib usr.sbin usr.bin; do + potential_directory="${potential_parent}/${contrib_dir}" if [ -d "${potential_directory}" ]; then echo "${potential_directory}" fi + done done + # Not automatically detected in the loop above + # because the contrib directories are named + # differently. + echo "usr.bin/svn" + # These files aren't useful without sendmail itself. # More importantly, some of them are licensed under the # "Sendmail License" which isn't part of the directory. -- 2.37.1 From 83416f75683a5395149284b7e2287c367b4e5ca1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 04:38:24 +0200 Subject: [PATCH 170/310] release/scripts/strip-freebsd.sh: Ditch the liquidio firmware ... which was added in r323509/5543e587c7a (not yet MFC'd) with unclear license conditions. A modified version of the supplised MK_SOURCELESS_UCODE fix from https://lists.freebsd.org/pipermail/svn-src-all/2017-September/151110.html was committed in r323543/0091cace28a, but the question about the license remains without reply (2017-09-18). Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index b8be6b84ec9d..a394bb16792f 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -79,7 +79,7 @@ get_unsupported_architectures() { # These depend on or contain proprietary firmware that is included in sys/contrib/dev get_tainted_sys_contrib_devs() { - echo "drm2 ipw iwi iwm iwn mwl npe otus ral rsu rtwn run uath urtwn wpi" + echo "drm2 ipw iwi iwm iwn liquidio mwl npe otus ral rsu rtwn run uath urtwn wpi" } # These require proprietary firmware that is included in sys/dev -- 2.37.1 From c8eb5b4d802c55d93ff55cb65c93496de51f0368 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 15 Sep 2017 13:17:05 +0200 Subject: [PATCH 171/310] release/scripts/strip-freebsd.sh: Add tdfx to the list of tainted sys devs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TL;DR: gesehen, gelacht, gelöscht Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index a394bb16792f..4b15c8e87f99 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -88,6 +88,20 @@ get_tainted_sys_devs() { # bce has already been taken care of by get_bce_files() above echo "bxe ctau cx cxgb cxgbe ispfw qlxgbe qlnx" \ "it tw" + # known tdfx issues: + # tdfx was supposedly "developed by Coleman Kane" but the license + # text mentions "Gardner Buchanan" in clause 3 and 4 without + # a matching "Copyright" line. Did somebody copy and paste the + # license text from sys/dev/sn without reading it? + # + # Also one of tdfx's header files "is basically a derivative of + # some sys/dhio.h file" which indicates that "some" other file's + # license may be relevant here. + # + # Luckily "3Dfx Voodoo" support isn't too relevant for most ElectroBSD + # use cases so figuring out the license problems is a low priority issue + # because we can simply ditch the offending code. + echo "tdfx" } get_unused_contrib_dirs() { -- 2.37.1 From e4d363aa6a6911d6c50ed6905f73d5050e4f48d7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 23 Aug 2017 08:50:43 +0200 Subject: [PATCH 172/310] share/dtrace: Import bio-request-latency Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/bio-request-latency | 116 +++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+) create mode 100755 share/dtrace/bio-request-latency diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 6efaf27d1304..2c0f8963a920 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -7,6 +7,7 @@ PACKAGE= dtrace SCRIPTS= blocking \ + bio-request-latency \ disklatency \ disklatencycmd \ fbt-time \ diff --git a/share/dtrace/bio-request-latency b/share/dtrace/bio-request-latency new file mode 100755 index 000000000000..be107a88edf2 --- /dev/null +++ b/share/dtrace/bio-request-latency @@ -0,0 +1,116 @@ +#!/usr/sbin/dtrace -s +/*************************************************************************** + * bio-request-latency.d + * + * Shows the latency for each disk and BIO type. + * + * io probes without BIO are ignored as they don't provide + * enough information. For details see: + * https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220965 + * + * Inspired by: + * http://dtrace.org/blogs/ahl/2014/08/31/openzfs-tuning/ + * + * Copyright (c) 2017 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +BEGIN +{ + start = timestamp; + bogus_timestamps = 0; + + bio_command[0x01] = "BIO_READ"; + bio_command[0x02] = "BIO_WRITE"; + bio_command[0x03] = "BIO_DELETE"; + bio_command[0x04] = "BIO_GETATTR"; + bio_command[0x05] = "BIO_FLUSH"; + bio_command[0x06] = "BIO_CMD0"; + bio_command[0x07] = "BIO_CMD1"; + bio_command[0x08] = "BIO_CMD2"; + bio_command[0x09] = "BIO_ZONE"; +} + +io:::start +/arg0 != NULL/ +{ + io_start[arg0] = timestamp; +} + +/* Cache timestamp so we can sanity check it */ +io:::done +{ + this->timestamp = timestamp; +} + +io:::done +/arg0 != NULL && io_start[arg0] && io_start[arg0] > this->timestamp/ +{ + /* + * At least in theory the timestamps goes up, in practice + * timestamps can jump back due to bugs like: + * https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218452 + * + * If this happens the timestamp delta is obviously useless + * for our purposes so lets not include it in the stats. + */ + bogus_timestamps++; + io_start[arg0] = 0; +} + +/* If this section gets entered this->timestamp isn't obviouly bogus */ +io:::done +/arg0 != NULL && io_start[arg0]/ +{ + this->delta = (this->timestamp - io_start[arg0]) / 1000; + this->bio_command = (bio_command[args[0]->bio_cmd] != NULL) ? + bio_command[args[0]->bio_cmd] : "Unregistered command"; + this->disk_name = stringof(args[0]->bio_disk->d_geom->name); + + @quantize[this->disk_name, this->bio_command] = quantize(this->delta); + @average[this->disk_name, this->bio_command] = avg(this->delta); + @stddev[this->disk_name, this->bio_command] = stddev(this->delta); + @max[this->disk_name, this->bio_command] = max(this->delta); + @min[this->disk_name, this->bio_command] = min(this->delta); + @requests[this->disk_name, this->bio_command] = count(); + @ios[this->disk_name, this->bio_command] = count(); + @throughput[this->disk_name, this->bio_command] = sum(args[0]->bio_length); + + io_start[arg0] = 0; +} + +END +{ + printa(@quantize); + + normalize(@ios, (timestamp - start) / 1000000000); + normalize(@throughput, (timestamp - start) / 1000000000 * 1024); + + printf("%-8s %-12s %10s | %-15s %-23s | %6s %12s\n", + "Disk", "Bio Type", "Requests", "", "Latency in us", "iops", "Throughput"); + printf("%-32s | %9s %9s %9s %9s |\n", "", "avg", "stddev", "max", "min"); + printa("%-8s %-12s %@10u | %@9u %@9u %@9u %@9u | %@6u %@8u k/s\n", + @requests, @average, @stddev, @max, @min, @ios, @throughput); +} + +END +/bogus_timestamps != 0/ +{ + printf("\nLatency for %d requests not measured due to obviously bogus timestamps.\n", + bogus_timestamps); + printf("Note that some of the other measurements may be based on less-obviously bogus timestamps.\n"); +} -- 2.37.1 From 7d2fa210e4c291184948246ec2c1c793f1f6e2ff Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 23 Aug 2017 08:56:36 +0200 Subject: [PATCH 173/310] share/dtrace: Import zfs-txg-sync-info 2017-08-23-ed990899 Obtained from: ElectroBSD --- share/dtrace/Makefile | 3 +- share/dtrace/zfs-txg-sync-info | 58 ++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 1 deletion(-) create mode 100755 share/dtrace/zfs-txg-sync-info diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 2c0f8963a920..238e36efedd4 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -23,7 +23,8 @@ SCRIPTS= blocking \ tcpstate \ tcptrack \ udptrack \ - zfsdbg-msg + zfsdbg-msg \ + zfs-txg-sync-info SCRIPTSDIR= ${SHAREDIR}/dtrace diff --git a/share/dtrace/zfs-txg-sync-info b/share/dtrace/zfs-txg-sync-info new file mode 100755 index 000000000000..00f5614cac2c --- /dev/null +++ b/share/dtrace/zfs-txg-sync-info @@ -0,0 +1,58 @@ +#!/usr/sbin/dtrace -s +/*************************************************************************** + * zfs-txg-sync-info.d + * + * Shows the txg sync time and the amount of dirty data that got synced. + * By default all pools are monitored. If a pool name is specified as + * first argument, only that pool is monitored. + * + * Inspired by: + * http://dtrace.org/blogs/ahl/2014/08/31/openzfs-tuning/ + * + * Copyright (c) 2017 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option defaultargs + +BEGIN +{ + pool_to_watch = $$1; +} + +txg-syncing +/pool_to_watch == NULL || pool_to_watch == ((dsl_pool_t *)arg0)->dp_spa->spa_name/ +{ + this->pool_name = stringof(((dsl_pool_t *)arg0)->dp_spa->spa_name); + sync_start[this->pool_name] = timestamp; + dp_dirty_total[this->pool_name] = ((dsl_pool_t *)arg0)->dp_dirty_total; + dp_dirty_percentage[this->pool_name] = dp_dirty_total[this->pool_name] / (`zfs_dirty_data_max / 100); +} + +txg-synced +/this->pool_name != NULL/ +{ + this->sync_time = timestamp - sync_start[this->pool_name]; + + printf("%Y: %s txg written in %d.%02d seconds. %3d.%02d MB were dirty (%2d%% of %d MB)\n", + walltimestamp, + this->pool_name, + this->sync_time / 1000000000, + this->sync_time / 10000000 % 100, + dp_dirty_total[this->pool_name] / 1024 / 1024, + dp_dirty_total[this->pool_name] / 1024 % 100, + dp_dirty_percentage[this->pool_name], + `zfs_dirty_data_max / 1024 / 1024); +} -- 2.37.1 From 1282a12db7543eb86320e3f8e9cd31a5150b1b08 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 25 Aug 2017 10:30:37 +0200 Subject: [PATCH 174/310] Add share/dtrace/iosched-monitor (unfinished WIP) so it no longer gets listed as 'untracked' Obtained from: ElectroBSD --- share/dtrace/iosched-monitor | 87 ++++++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100755 share/dtrace/iosched-monitor diff --git a/share/dtrace/iosched-monitor b/share/dtrace/iosched-monitor new file mode 100755 index 000000000000..dca024a49dc7 --- /dev/null +++ b/share/dtrace/iosched-monitor @@ -0,0 +1,87 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * iosched-monitor + * + * Monitors the behavior of the Dynamic I/O scheduler. + * + * Copyright (c) 2017 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet + +dtrace:::BEGIN +{ + bio_command[0x01] = "BIO_READ"; + bio_command[0x02] = "BIO_WRITE"; + bio_command[0x03] = "BIO_DELETE"; + bio_command[0x04] = "BIO_GETATTR"; + bio_command[0x05] = "BIO_FLUSH"; + bio_command[0x06] = "BIO_CMD0"; + bio_command[0x07] = "BIO_CMD1"; + bio_command[0x08] = "BIO_CMD2"; + bio_command[0x09] = "BIO_ZONE"; +} + +fbt::cam_iosched_bio_complete:entry +{ + this->isc = args[0]; + this->bp = args[1]; + + this->bio_command = (bio_command[this->bp->bio_cmd] != NULL) ? + bio_command[this->bp->bio_cmd] : "Unregistered command"; +} + +fbt::cam_iosched_bio_complete:return +/this->isc && this->bio_command == "BIO_DELETE"/ +{ + printf("%Y: cam_iosched_bio_complete returned for BIO_DELETE. Pending trims: %d\n", + walltimestamp, this->isc->trim_stats.pending); +} + +fbt::cam_iosched_trim_done:entry +{ + this->isc = args[0]; + printf("%Y: cam_iosched_trim_done called. Pending trims: %d\n", + walltimestamp, this->isc->trim_stats.pending); +} + +fbt::cam_iosched_put_back_trim:entry +{ + this->isc = args[0]; +} + +fbt::cam_iosched_put_back_trim:return +/this->isc/ +{ + printf("%Y: cam_iosched_put_back returned. Pending trims: %d\n", + walltimestamp, this->isc->trim_stats.pending); +} + +fbt::cam_iosched_next_trim:entry +{ + this->isc = args[0]; +} + +fbt::cam_iosched_next_trim:return +/this->isc/ +{ + printf("%Y: cam_iosched_next_trim returned. Pending trims: %d\n", + walltimestamp, this->isc->trim_stats.pending); +} + +END +{ +} -- 2.37.1 From 1552d02279174b9efcb81af68984b0bacfbe87df Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Aug 2017 18:44:43 +0200 Subject: [PATCH 175/310] share/dtrace: Import flowtrace 2017-01-28/d600753af03 Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/flowtrace | 165 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 166 insertions(+) create mode 100755 share/dtrace/flowtrace diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 238e36efedd4..bee198852f09 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -11,6 +11,7 @@ SCRIPTS= blocking \ disklatency \ disklatencycmd \ fbt-time \ + flowtrace \ geli-key-monitor \ geli-request-monitor \ hotopen \ diff --git a/share/dtrace/flowtrace b/share/dtrace/flowtrace new file mode 100755 index 000000000000..9e890f61a311 --- /dev/null +++ b/share/dtrace/flowtrace @@ -0,0 +1,165 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * flowtrace.d + * + * Enables flow tracing after entering the function specified as + * first argument and disables it again after the function returns. + * + * For a number of currently-hardcoded functions the children + * don't get shown in the output to make it easier to comprehend. + * + * XXX: It would be more efficient to not generate probes for them. + * This probably requires another script that generates the D + * code on the fly. + * + * Copyright (c) 2012 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option defaultargs +#pragma D option dynvarsize=10m +#pragma D option bufsize=50m +#pragma D option switchrate=10hz +#pragma D option quiet + +dtrace:::BEGIN +{ + /* + * Initialize indent[] which is used to indent the probefunc + * according to the stackdepth. Unlike "#pragma D option flowindent" + * this doesn't break in case of missing return probes. + * + * The rest of the values are filled in the dtrace::BEGIN probe + * at the end of the script. + */ + indent[0] = ""; + + traced_function = $$1; + + this->timestamp = walltimestamp; + this->msecs = (this->timestamp / 1000000) % 1000; + printf("%d %d %Y.%.3d 00: Trace in progress. Waiting to enter %s. Hit CTRL-C to exit.\n", + cpu, tid, this->timestamp, this->msecs, traced_function); +} + +/* Prepare to trace every function on this thread from now on ... */ +fbt::$$1:entry +{ + self->trace_me_yo = 1; +} + +/* ... except for the children of these functions. */ +fbt::hpet_intr:return, +fbt::lapic_handle_intr:return, +fbt::termcn_cnputc:return, +fbt::printf:return, +fbt::cv_signal:return, +fbt::sleepq_broadcast:return +/self->trace_me_yo && self->stfu == stackdepth/ +{ + self->stfu = 0; +} + +/* This is where the actual tracing happens. */ +fbt::: +/self->trace_me_yo && ! self->stfu/ +{ + this->timestamp = walltimestamp; + this->msecs = (this->timestamp / 1000000) % 1000; + this->direction = probename == "entry" ? "-->" : "<--"; + printf("%d %d %Y.%.3d %.2d: %s %s %s:%s\n", + cpu, tid, this->timestamp, this->msecs, + stackdepth, indent[stackdepth], + this->direction, probefunc, probename); +} + +fbt::hpet_intr:entry, +fbt::lapic_handle_intr:entry, +fbt::termcn_cnputc:entry, +fbt::printf:entry, +fbt::cv_signal:entry, +fbt::sleepq_broadcast:entry +/self->trace_me_yo && !self->stfu/ +{ + this->timestamp = walltimestamp; + this->msecs = (this->timestamp / 1000000) % 1000; + printf("%d %d %Y.%.3d %.2d: %s [...]\n", + cpu, tid, this->timestamp, this->msecs, + stackdepth, indent[stackdepth]); + self->stfu = stackdepth; +} + +/* Done tracing */ +fbt::$$1:return +{ + self->trace_me_yo = 0; + exit(0); +} + +dtrace:::BEGIN +{ + /* perl -e 'for my $i (0..50) {printf(" indent[%s%d] = \"%s\";\n", $i < 10 ? " " : "", $i, " " x $i);}' */ + indent[ 0] = ""; + indent[ 1] = " "; + indent[ 2] = " "; + indent[ 3] = " "; + indent[ 4] = " "; + indent[ 5] = " "; + indent[ 6] = " "; + indent[ 7] = " "; + indent[ 8] = " "; + indent[ 9] = " "; + indent[10] = " "; + indent[11] = " "; + indent[12] = " "; + indent[13] = " "; + indent[14] = " "; + indent[15] = " "; + indent[16] = " "; + indent[17] = " "; + indent[18] = " "; + indent[19] = " "; + indent[20] = " "; + indent[21] = " "; + indent[22] = " "; + indent[23] = " "; + indent[24] = " "; + indent[25] = " "; + indent[26] = " "; + indent[27] = " "; + indent[28] = " "; + indent[29] = " "; + indent[30] = " "; + indent[31] = " "; + indent[32] = " "; + indent[33] = " "; + indent[34] = " "; + indent[35] = " "; + indent[36] = " "; + indent[37] = " "; + indent[38] = " "; + indent[39] = " "; + indent[40] = " "; + indent[41] = " "; + indent[42] = " "; + indent[43] = " "; + indent[44] = " "; + indent[45] = " "; + indent[46] = " "; + indent[47] = " "; + indent[48] = " "; + indent[49] = " "; + indent[50] = " "; +} -- 2.37.1 From 5cca791420a94d44fa3bfc9129dfea22ec5eb83f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 22 Aug 2017 10:01:08 +0200 Subject: [PATCH 176/310] sys/geom/eli: Ignore the passphrase set in the loader if it doesn't contain a single character Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 529e0f006224..1da042018145 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -112,9 +112,24 @@ fetch_loader_passphrase(void * dummy) KASSERT(dynamic_kenv, ("need dynamic kenv")); if ((env_passphrase = kern_getenv("kern.geom.eli.passphrase")) != NULL) { - /* Extract passphrase from the environment. */ - strlcpy(cached_passphrase, env_passphrase, - sizeof(cached_passphrase)); + if (env_passphrase[0] != '\0') { + /* Extract passphrase from the environment. */ + strlcpy(cached_passphrase, env_passphrase, + sizeof(cached_passphrase)); + G_ELI_DEBUG(0, "Using cached passphrase!"); + } else { + /* + * Passphrase is set in the environment but empty. + * This can happen if the user actually uses an empty + * passphrase (unlikely) and if the user pressed return + * to skip the passphrase prompt in the loader. + * + * In the second case we do not want to delay the boot + * by iterating over the empty and bogus passphrase and + * ask for another passphrase right away. + */ + G_ELI_DEBUG(0, "Ignoring cached passphrase as it seems to be empty!"); + } freeenv(env_passphrase); /* Wipe the passphrase from the environment. */ -- 2.37.1 From 5fdf2cda23ea19500c7dad201dc0d6c9c4614b47 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 22 Aug 2017 11:18:26 +0200 Subject: [PATCH 177/310] sys/geom/eli: Show the environment passphrase if the visible_passphrase sysctl is set Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 1da042018145..47abbb769a76 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -112,6 +112,8 @@ fetch_loader_passphrase(void * dummy) KASSERT(dynamic_kenv, ("need dynamic kenv")); if ((env_passphrase = kern_getenv("kern.geom.eli.passphrase")) != NULL) { + if (g_eli_visible_passphrase) + G_ELI_DEBUG(0, "Passphrase is %s", env_passphrase); if (env_passphrase[0] != '\0') { /* Extract passphrase from the environment. */ strlcpy(cached_passphrase, env_passphrase, -- 2.37.1 From b654b3368d8b7e36e9f6fb73fc7150a4969a13d8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 11:56:04 +0200 Subject: [PATCH 178/310] acpi_ibm: Disable Bluetooth by default without looking how the hardware is inititialized Obtained from: ElectroBSD --- sys/dev/acpi_support/acpi_ibm.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sys/dev/acpi_support/acpi_ibm.c b/sys/dev/acpi_support/acpi_ibm.c index 84b17dbe9bdd..26f514abc36a 100644 --- a/sys/dev/acpi_support/acpi_ibm.c +++ b/sys/dev/acpi_support/acpi_ibm.c @@ -1019,6 +1019,10 @@ acpi_ibm_sysctl_init(struct acpi_ibm_softc *sc, int method) return (FALSE); case ACPI_IBM_METHOD_BLUETOOTH: + /* Disable bluetooth by default. */ + acpi_ibm_bluetooth_set(sc, 0); + + /* fallthrough */ case ACPI_IBM_METHOD_WLAN: if (ACPI_SUCCESS(acpi_GetInteger(sc->handle, IBM_NAME_WLAN_BT_GET, &dummy))) return (TRUE); -- 2.37.1 From 6d1757679d9fceb5b8c54a92c139f389f36170d4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 13:35:13 +0200 Subject: [PATCH 179/310] tools/build/options/makeman: Add silly hack to deal with options that have no description (XXX) If the option doesn't have a description file it's likely that the opposite option does have one. If it does, use its opossit and hope for the best. XXX: Does not confirm that the meaning of the description has actually been reverted by the sed calls. Obtained from: ElectroBSD --- tools/build/options/makeman | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/tools/build/options/makeman b/tools/build/options/makeman index 328b9faed881..ea544d542dd4 100755 --- a/tools/build/options/makeman +++ b/tools/build/options/makeman @@ -249,13 +249,26 @@ EOF show_options | while read opt targets ; do - if [ ! -f ${opt} ] ; then - echo "no description found for ${opt}, skipping" >&2 - continue + echo ".It Va ${opt}" + + if [ -f ${opt} ] ; then + sed -e'/\$FreeBSD.*\$/d' ${opt} + else + warning="no description found for ${opt}" + opposit_opt=WITH${opt#WITHOUT} + if [ -f ${opposit_opt} ] ; then + warning="${warning}. Using the opossite of ${opposit_opt}'s description" >&2 + sed -e'/\$FreeBSD.*\$/d' -E -e 's@(Set to) (build)@\1 not \2@' ${opposit_opt} + else + opposit_opt=WITHOUT${opt#WITH} + if [ -f ${opposit_opt} ] ; then + warning="${warning}. Using opossite of ${opposit_opt}'s description" >&2 + sed -e'/\$FreeBSD.*\$/d' -E -e 's@(Set to) not (build)@\1 \2@' ${opposit_opt} + fi + fi + echo "${warning}" >&2 fi - echo ".It Va ${opt}" - sed -e'/\$FreeBSD.*\$/d' ${opt} if [ -n "${targets}" ] ; then echo '.Pp' echo 'This is a default setting on' -- 2.37.1 From 179dfc8d26552cea2fe936f971c0132f0fef0299 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 13:40:58 +0200 Subject: [PATCH 180/310] Makefile: Explicitly reduce TARGETS to amd64 and i386 (XXX: workaround) Other targets currently aren't supported by ElectroBSD and somehow result in /usr/src/tools/build/options/makeman entering an endless loop. Reducing the targets doesn't impact the supported targets and works around the "makeman" issue for "now". Obtained from: ElectroBSD --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index e16557f83371..735c0feafa82 100644 --- a/Makefile +++ b/Makefile @@ -516,7 +516,7 @@ EXTRA_ARCHES_mips+= mipsn32 # powerpcspe excluded from main list until clang fixed EXTRA_ARCHES_powerpc= powerpcspe powerpc64le .endif -TARGETS?=amd64 arm arm64 i386 mips powerpc riscv +TARGETS?=amd64 i386 _UNIVERSE_TARGETS= ${TARGETS} TARGET_ARCHES_arm?= armv6 armv7 TARGET_ARCHES_arm64?= aarch64 -- 2.37.1 From f7aba42c975ddbb7d8a630585c5eaba01b9579e8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 Sep 2017 16:41:47 +0200 Subject: [PATCH 181/310] zfs: Allow to set zfs_send_set_freerecords_bit at runtime Obtained from: ElectroBSD --- sys/contrib/openzfs/module/zfs/dmu_send.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/contrib/openzfs/module/zfs/dmu_send.c b/sys/contrib/openzfs/module/zfs/dmu_send.c index 390afba92680..a96ddcae916d 100644 --- a/sys/contrib/openzfs/module/zfs/dmu_send.c +++ b/sys/contrib/openzfs/module/zfs/dmu_send.c @@ -101,6 +101,8 @@ int zfs_override_estimate_recordsize = 0; /* Set this tunable to FALSE to disable setting of DRR_FLAG_FREERECORDS */ int zfs_send_set_freerecords_bit = B_TRUE; +ZFS_MODULE_PARAM(zfs, zfs_, send_set_freerecords_bit, INT, ZMOD_RW, + "Set the freerecords bits when sending"); /* Set this tunable to FALSE is disable sending unmodified spill blocks. */ int zfs_send_unmodified_spill_blocks = B_TRUE; -- 2.37.1 From ab0c98d0eda0cd47153409248b05043fbac86a00 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 13 Nov 2017 21:41:37 +0100 Subject: [PATCH 182/310] i386: Don't try to build 3dfx as it gets removed by strip-freebsd.sh (XXX: _aic and _apm) Obtained from: ElectroBSD --- sys/modules/Makefile | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index a8dffd45cb4f..ffd0cf9b0427 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -810,8 +810,12 @@ _vmm= vmm .if ${MACHINE_CPUARCH} == "i386" # XXX some of these can move to the general case when de-i386'ed # XXX some of these can move now, but are untested on other architectures. -_3dfx= 3dfx -_3dfx_linux= 3dfx_linux +# +# license unclear +#_3dfx= 3dfx +#_3dfx_linux= 3dfx_linux +_aic= aic +_apm= apm .if ${MK_SOURCELESS_HOST} != "no" _ce= ce .endif -- 2.37.1 From 16a636c866bf8a470b1792c5dbe916b0ed5a06fb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 14 Nov 2017 22:28:25 +0100 Subject: [PATCH 183/310] cam: Fix nullpointer dereference Unread portion of the kernel message buffer: ahcich1: Timeout on slot 27 port 0 ahcich1: is 00000000 cs 08000000 ss 00000000 rs 08000000 tfd 451 serr 00000000 cmd 000c5a17 Fatal trap 12: page fault while in kernel mode (aprobe0:ahcich1:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 00 40 00 00 00 00 08 00 cpuid = 2; apic id = 02 fault virtual address = 0x28 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff802a3981 stack pointer = 0x28:0xfffffe01ea3747e0 frame pointer = 0x28:0xfffffe01ea374810 (aprobe0:ahcich1:0:0:0): CAM status: Command timeout (aprobe0:ahcich1:0:0:0): Retrying command code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi4: clock (0)) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01ea3743c0 vpanic() at vpanic+0x186/frame 0xfffffe01ea374440 panic() at panic+0x43/frame 0xfffffe01ea3744a0 trap_fatal() at trap_fatal+0x34d/frame 0xfffffe01ea3744f0 trap_pfault() at trap_pfault+0x49/frame 0xfffffe01ea374550 trap() at trap+0x29a/frame 0xfffffe01ea374710 calltrap() at calltrap+0x8/frame 0xfffffe01ea374710 --- trap 0xc, rip = 0xffffffff802a3981, rsp = 0xfffffe01ea3747e0, rbp = 0xfffffe01ea374810 --- xpt_async() at xpt_async+0x331/frame 0xfffffe01ea374810 ahci_reset() at ahci_reset+0x1fe/frame 0xfffffe01ea374860 ahci_end_transaction() at ahci_end_transaction+0x8bd/frame 0xfffffe01ea3748c0 ahci_timeout() at ahci_timeout+0x334/frame 0xfffffe01ea374900 softclock_call_cc() at softclock_call_cc+0x13b/frame 0xfffffe01ea3749b0 softclock() at softclock+0xb9/frame 0xfffffe01ea3749e0 intr_event_execute_handlers() at intr_event_execute_handlers+0xec/frame 0xfffffe01ea374a20 ithread_loop() at ithread_loop+0xd6/frame 0xfffffe01ea374a70 fork_exit() at fork_exit+0x85/frame 0xfffffe01ea374ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01ea374ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 2h57m40s Dumping 1433 out of 8055 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% [...] __curthread () at ./machine/pcpu.h:222 222 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) where #0 __curthread () at ./machine/pcpu.h:222 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:298 #2 0xffffffff80579176 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #3 0xffffffff80579650 in vpanic (fmt=, ap=0xfffffe01ea374480) at /usr/src/sys/kern/kern_shutdown.c:759 #4 0xffffffff80579483 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #5 0xffffffff80855f6d in trap_fatal (frame=0xfffffe01ea374720, eva=40) at /usr/src/sys/amd64/amd64/trap.c:799 #6 0xffffffff80855fc9 in trap_pfault (frame=0xfffffe01ea374720, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:653 #7 0xffffffff8085581a in trap (frame=0xfffffe01ea374720) at /usr/src/sys/amd64/amd64/trap.c:420 #8 #9 0xffffffff802a3981 in xpt_async (async_code=1, path=, async_arg=0x0) at /usr/src/sys/cam/cam_xpt.c:4295 #10 0xffffffff8034d40e in ahci_reset (ch=0xfffffe0000d0c000) at /usr/src/sys/dev/ahci/ahci.c:2343 #11 0xffffffff8034ce2d in ahci_end_transaction (slot=, et=) at /usr/src/sys/dev/ahci/ahci.c:1957 #12 0xffffffff8034ea74 in ahci_process_timeout (ch=) at /usr/src/sys/dev/ahci/ahci.c:1671 #13 ahci_timeout (slot=0xfffffe0000d0ccb0) at /usr/src/sys/dev/ahci/ahci.c:1768 #14 0xffffffff805913ab in softclock_call_cc (c=, cc=0xffffffff80f08c80 , direct=) at /usr/src/sys/kern/kern_timeout.c:729 #15 0xffffffff80591909 in softclock (arg=0xffffffff80f08c80 ) at /usr/src/sys/kern/kern_timeout.c:867 #16 0xffffffff8053e89c in intr_event_execute_handlers (p=, ie=0xfffff8000326f500) at /usr/src/sys/kern/kern_intr.c:1262 #17 0xffffffff8053ef66 in ithread_execute_handlers (ie=, p=) at /usr/src/sys/kern/kern_intr.c:1275 #18 ithread_loop (arg=0xfffff800032487e0) at /usr/src/sys/kern/kern_intr.c:1356 #19 0xffffffff8053be85 in fork_exit (callout=0xffffffff8053ee90 , arg=0xfffff800032487e0, frame=0xfffffe01ea374ac0) at /usr/src/sys/kern/kern_fork.c:1044 #20 (kgdb) f 9 #9 0xffffffff802a3981 in xpt_async (async_code=1, path=, async_arg=0x0) at /usr/src/sys/cam/cam_xpt.c:4295 warning: Source file is more recent than executable. 4295 xpt_freeze_simq(path->bus->sim, 1); (kgdb) p path $1 = (kgdb) l - 4290 ccb->casync.async_arg_size = size; 4291 } 4292 if (path->device != NULL && path->device->lun_id != CAM_LUN_WILDCARD) 4293 xpt_freeze_devq(path, 1); 4294 else 4295 xpt_freeze_simq(path->bus->sim, 1); 4296 xpt_done(ccb); 4297 } 4298 4299 static void (kgdb) p path $2 = (kgdb) l - 4280 xpt_print(path, "Can't allocate argument to send %s\n", 4281 xpt_async_string(async_code)); 4282 xpt_free_path(ccb->ccb_h.path); 4283 xpt_free_ccb(ccb); 4284 return; 4285 } 4286 memcpy(ccb->casync.async_arg_ptr, async_arg, size); 4287 ccb->casync.async_arg_size = size; 4288 } else if (size < 0) { 4289 ccb->casync.async_arg_ptr = async_arg; (kgdb) l - 4270 size = xpt_async_size(async_code); 4271 CAM_DEBUG(ccb->ccb_h.path, CAM_DEBUG_TRACE, 4272 ("xpt_async: func %#x %s aync_code %d %s\n", 4273 ccb->ccb_h.func_code, 4274 xpt_action_name(ccb->ccb_h.func_code), 4275 async_code, 4276 xpt_async_string(async_code))); 4277 if (size > 0 && async_arg != NULL) { 4278 ccb->casync.async_arg_ptr = malloc(size, M_CAMXPT, M_NOWAIT); 4279 if (ccb->casync.async_arg_ptr == NULL) { (kgdb) f 10 #10 0xffffffff8034d40e in ahci_reset (ch=0xfffffe0000d0c000) at /usr/src/sys/dev/ahci/ahci.c:2343 2343 xpt_async(AC_BUS_RESET, ch->path, NULL); (kgdb) p ch->path $3 = (struct cam_path *) 0xfffff800032d3c80 (kgdb) p *ch->path $4 = {periph = 0x0, bus = 0x0, target = 0x0, device = 0x0} (kgdb) p *ch $5 = {dev = 0xfffff80003280b00, unit = 1, r_mem = 0xfffff80003474280, r_irq = 0xfffff80003474500, ih = 0x0, dma = {work_tag = 0xfffff80002fc8500, work_map = 0x0, work = 0xfffffe0233629000 "E", work_bus = 79859712, rfis_tag = 0xfffff80002fc8400, rfis_map = 0x0, rfis = 0xfffff80002fc8300 "\250\331\360YK\357\032q)\236\327\227\377^\332 -\341\260\300\356!J\237\065hr\304@a\\\036_`X", rfis_bus = 50103040, data_tag = 0xfffff80002fc8200}, sim = 0xfffff80003280400, path = 0xfffff800032d3c80, caps = 4281401189, caps2 = 4, chcaps = 786436, chscaps = 0, vendorid = 32902, deviceid = 7171, subvendorid = 6058, subdeviceid = 8655, quirks = 0, numslots = 32, pm_level = 0, devices = 1, pm_present = 0, fbs_enabled = 0, start = 0x0, hold = {0x0 }, slot = {{ch = 0xfffffe0000d0c000, slot = 0 '\000', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7ea00, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = { sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44295607766411, c_precision = 4026531562, c_arg = 0xfffffe0000d0c1b8, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 1 '\001', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e980, nsegs = 1}, timeout = {c_links = {le = {le_next = 0xfffffe0000d0ce10, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffffe0000d0ce10}, tqe = { tqe_next = 0xfffffe0000d0ce10, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45074001815689, c_precision = 4026531562, c_arg = 0xfffffe0000d0c220, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 2 '\002', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e900, nsegs = 0}, timeout = { c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45139272429188, c_precision = 4026531562, c_arg = 0xfffffe0000d0c288, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, { ch = 0xfffffe0000d0c000, slot = 3 '\003', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e880, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff800035bb3a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff800035bb3a0}, tqe = {tqe_next = 0xfffff800035bb3a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45203735588891, c_precision = 4026531562, c_arg = 0xfffffe0000d0c2f0, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 4 '\004', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e800, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = { tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44360878379910, c_precision = 4026531562, c_arg = 0xfffffe0000d0c358, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 5 '\005', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e780, nsegs = 0}, timeout = { c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44425332949679, c_precision = 4026531562, c_arg = 0xfffffe0000d0c3c0, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 6 '\006', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e700, nsegs = 1}, timeout = {c_links = {le = {le_next = 0xfffffe0000d0c318, le_prev = 0xffffffff80f08d98 }, sle = { sle_next = 0xfffffe0000d0c318}, tqe = {tqe_next = 0xfffffe0000d0c318, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45203735588891, c_precision = 4026531562, c_arg = 0xfffffe0000d0c428, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 7 '\a', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e680, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45268997612456, c_precision = 4026531562, c_arg = 0xfffffe0000d0c490, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 8 '\b', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e600, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45333439297324, c_precision = 4026531562, c_arg = 0xfffffe0000d0c4f8, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 9 '\t', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e580, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = { sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44490607858145, c_precision = 4026531562, c_arg = 0xfffffe0000d0c560, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 10 '\n', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e500, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff800035bb3a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff800035bb3a0}, tqe = { tqe_next = 0xfffff800035bb3a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44555071017848, c_precision = 4026531562, c_arg = 0xfffffe0000d0c5c8, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 11 '\v', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e480, nsegs = 1}, timeout = { c_links = {le = {le_next = 0xfffffe0000d0c520, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffffe0000d0c520}, tqe = {tqe_next = 0xfffffe0000d0c520, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45333443592291, c_precision = 4026531562, c_arg = 0xfffffe0000d0c630, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, { ch = 0xfffffe0000d0c000, slot = 12 '\f', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e400, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45398744270559, c_precision = 4026531562, c_arg = 0xfffffe0000d0c698, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 13 '\r', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e380, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff8001b2293a0}, tqe = { tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45463190250394, c_precision = 4026531562, c_arg = 0xfffffe0000d0c700, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 14 '\016', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e300, nsegs = 0}, timeout = { c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44620333041413, c_precision = 4026531562, c_arg = 0xfffffe0000d0c768, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, { ch = 0xfffffe0000d0c000, slot = 15 '\017', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e280, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff800035bb3a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff800035bb3a0}, tqe = {tqe_next = 0xfffff800035bb3a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44684796201116, c_precision = 4026531562, c_arg = 0xfffffe0000d0c7d0, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 16 '\020', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e200, nsegs = 1}, timeout = {c_links = {le = {le_next = 0xfffffe0000d0c728, le_prev = 0xffffffff80f08d98 }, sle = { sle_next = 0xfffffe0000d0c728}, tqe = {tqe_next = 0xfffffe0000d0c728, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45463190250394, c_precision = 4026531562, c_arg = 0xfffffe0000d0c838, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 17 '\021', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e180, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45528469453827, c_precision = 4026531562, c_arg = 0xfffffe0000d0c8a0, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 18 '\022', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e100, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45592924023596, c_precision = 4026531562, c_arg = 0xfffffe0000d0c908, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 19 '\023', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e080, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff800035bb3a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff800035bb3a0}, tqe = { ---Type to continue, or q to quit---qq tqe_next Quit (kgdb) where #0 __curthread () at ./machine/pcpu.h:222 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:298 #2 0xffffffff80579176 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #3 0xffffffff80579650 in vpanic (fmt=, ap=0xfffffe01ea374480) at /usr/src/sys/kern/kern_shutdown.c:759 #4 0xffffffff80579483 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #5 0xffffffff80855f6d in trap_fatal (frame=0xfffffe01ea374720, eva=40) at /usr/src/sys/amd64/amd64/trap.c:799 #6 0xffffffff80855fc9 in trap_pfault (frame=0xfffffe01ea374720, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:653 #7 0xffffffff8085581a in trap (frame=0xfffffe01ea374720) at /usr/src/sys/amd64/amd64/trap.c:420 #8 #9 0xffffffff802a3981 in xpt_async (async_code=1, path=, async_arg=0x0) at /usr/src/sys/cam/cam_xpt.c:4295 #10 0xffffffff8034d40e in ahci_reset (ch=0xfffffe0000d0c000) at /usr/src/sys/dev/ahci/ahci.c:2343 #11 0xffffffff8034ce2d in ahci_end_transaction (slot=, et=) at /usr/src/sys/dev/ahci/ahci.c:1957 #12 0xffffffff8034ea74 in ahci_process_timeout (ch=) at /usr/src/sys/dev/ahci/ahci.c:1671 #13 ahci_timeout (slot=0xfffffe0000d0ccb0) at /usr/src/sys/dev/ahci/ahci.c:1768 #14 0xffffffff805913ab in softclock_call_cc (c=, cc=0xffffffff80f08c80 , direct=) at /usr/src/sys/kern/kern_timeout.c:729 #15 0xffffffff80591909 in softclock (arg=0xffffffff80f08c80 ) at /usr/src/sys/kern/kern_timeout.c:867 #16 0xffffffff8053e89c in intr_event_execute_handlers (p=, ie=0xfffff8000326f500) at /usr/src/sys/kern/kern_intr.c:1262 #17 0xffffffff8053ef66 in ithread_execute_handlers (ie=, p=) at /usr/src/sys/kern/kern_intr.c:1275 #18 ithread_loop (arg=0xfffff800032487e0) at /usr/src/sys/kern/kern_intr.c:1356 #19 0xffffffff8053be85 in fork_exit (callout=0xffffffff8053ee90 , arg=0xfffff800032487e0, frame=0xfffffe01ea374ac0) at /usr/src/sys/kern/kern_fork.c:1044 #20 Obtained from: ElectroBSD --- sys/cam/cam_xpt.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/sys/cam/cam_xpt.c b/sys/cam/cam_xpt.c index 184a6b69b488..726dcca2d5ba 100644 --- a/sys/cam/cam_xpt.c +++ b/sys/cam/cam_xpt.c @@ -4393,10 +4393,12 @@ xpt_async(u_int32_t async_code, struct cam_path *path, void *async_arg) ccb->casync.async_arg_ptr = async_arg; ccb->casync.async_arg_size = size; } - if (path->device != NULL && path->device->lun_id != CAM_LUN_WILDCARD) - xpt_freeze_devq(path, 1); - else - xpt_freeze_simq(path->bus->sim, 1); + if (path != NULL) { + if (path->device != NULL && path->device->lun_id != CAM_LUN_WILDCARD) + xpt_freeze_devq(path, 1); + else + xpt_freeze_simq(path->bus->sim, 1); + } xpt_action(ccb); } -- 2.37.1 From 7adaf74d34008494d365cc00a0289667b68eaf36 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 14 Nov 2017 22:23:50 +0100 Subject: [PATCH 184/310] umass: Prevent nullpointer dereference in xpt_freeze_devq() Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x28 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff802a2ed2 stack pointer = 0x28:0xfffffe023416b900 frame pointer = 0x28:0xfffffe023416b920 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 96046 (usbus1) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe023416b4e0 vpanic() at vpanic+0x186/frame 0xfffffe023416b560 panic() at panic+0x43/frame 0xfffffe023416b5c0 trap_fatal() at trap_fatal+0x322/frame 0xfffffe023416b610 trap_pfault() at trap_pfault+0x49/frame 0xfffffe023416b670 trap() at trap+0x298/frame 0xfffffe023416b830 calltrap() at calltrap+0x8/frame 0xfffffe023416b830 --- trap 0xc, rip = 0xffffffff802a2ed2, rsp = 0xfffffe023416b900, rbp = 0xfffffe023416b920 --- xpt_freeze_devq() at xpt_freeze_devq+0x12/frame 0xfffffe023416b920 umass_cam_cb() at umass_cam_cb+0x163/frame 0xfffffe023416b940 umass_t_bbb_status_callback() at umass_t_bbb_status_callback+0x527/frame 0xfffffe023416b9a0 usbd_callback_wrapper() at usbd_callback_wrapper+0x63f/frame 0xfffffe023416b9f0 usb_command_wrapper() at usb_command_wrapper+0x10c/frame 0xfffffe023416ba10 usb_callback_proc() at usb_callback_proc+0x76/frame 0xfffffe023416ba30 usb_process() at usb_process+0xd9/frame 0xfffffe023416ba70 fork_exit() at fork_exit+0x85/frame 0xfffffe023416bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe023416bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 15h52m32s Dumping 1548 out of 8055 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% [...] __curthread () at ./machine/pcpu.h:222 222 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) where #0 __curthread () at ./machine/pcpu.h:222 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:298 #2 0xffffffff80579b16 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #3 0xffffffff80579ff0 in vpanic (fmt=, ap=0xfffffe023416b5a0) at /usr/src/sys/kern/kern_shutdown.c:759 #4 0xffffffff80579e23 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #5 0xffffffff80853172 in trap_fatal (frame=0xfffffe023416b840, eva=40) at /usr/src/sys/amd64/amd64/trap.c:799 #6 0xffffffff808531c9 in trap_pfault (frame=0xfffffe023416b840, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:653 #7 0xffffffff80852a48 in trap (frame=0xfffffe023416b840) at /usr/src/sys/amd64/amd64/trap.c:420 #8 #9 xpt_freeze_devq (path=0xfffff80002fbac80, count=1) at /usr/src/sys/cam/cam_xpt.c:4340 #10 0xffffffff816649f3 in umass_cam_cb (sc=0xfffff800b8a28800, ccb=0xfffff80114f5d800, residue=0, status=3 '\003') at /usr/src/sys/dev/usb/storage/umass.c:2519 #11 0xffffffff816633c7 in umass_cancel_ccb (sc=) at /usr/src/sys/dev/usb/storage/umass.c:1151 #12 umass_tr_error (xfer=, error=) at /usr/src/sys/dev/usb/storage/umass.c:1167 #13 umass_t_bbb_status_callback (xfer=, error=) at /usr/src/sys/dev/usb/storage/umass.c:1568 #14 0xffffffff8163cf9f in usbd_callback_wrapper (pq=) at /usr/src/sys/dev/usb/usb_transfer.c:2408 #15 0xffffffff8163e06c in usb_command_wrapper (pq=0xfffffe002e97d060, xfer=) at /usr/src/sys/dev/usb/usb_transfer.c:3062 #16 0xffffffff8163d1d6 in usb_callback_proc (_pm=) at /usr/src/sys/dev/usb/usb_transfer.c:2269 #17 0xffffffff81638359 in usb_process (arg=0xfffffe0008596db0) at /usr/src/sys/dev/usb/usb_process.c:176 #18 0xffffffff8053cc05 in fork_exit (callout=0xffffffff81638280 , arg=0xfffffe0008596db0, frame=0xfffffe023416bac0) at /usr/src/sys/kern/kern_fork.c:1043 #19 (kgdb) f 9 #9 xpt_freeze_devq (path=0xfffff80002fbac80, count=1) at /usr/src/sys/cam/cam_xpt.c:4340 warning: Source file is more recent than executable. 4340 devq = dev->sim->devq; (kgdb) p dev->sim Cannot access memory at address 0x28 (kgdb) p dev $1 = (struct cam_ed *) 0x0 (kgdb) f 10 #10 0xffffffff816649f3 in umass_cam_cb (sc=0xfffff800b8a28800, ccb=0xfffff80114f5d800, residue=0, status=3 '\003') at /usr/src/sys/dev/usb/storage/umass.c:2519 2519 xpt_freeze_devq(ccb->ccb_h.path, 1); (kgdb) p ccb->ccb_h.path $2 = (struct cam_path *) 0xfffff80002fbac80 (kgdb) p *ccb->ccb_h.path $3 = {periph = 0xfffff8019bbcb100, bus = 0x0, target = 0x0, device = 0x0} (kgdb) p *ccb->ccb_h.path.dev There is no member named dev. (kgdb) p *ccb->ccb_h.path->device Cannot access memory at address 0x0 (kgdb) p *ccb->ccb_h.path.device Cannot access memory at address 0x0 (kgdb) p ccb->ccb_h.path.device $4 = (struct cam_ed *) 0x0 (kgdb) p ccb->ccb_h.path->device $5 = (struct cam_ed *) 0x0 Obtained from: ElectroBSD --- sys/dev/usb/storage/umass.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sys/dev/usb/storage/umass.c b/sys/dev/usb/storage/umass.c index b8b502494264..6c96a54c21ef 100644 --- a/sys/dev/usb/storage/umass.c +++ b/sys/dev/usb/storage/umass.c @@ -2526,7 +2526,8 @@ umass_cam_cb(struct umass_softc *sc, union ccb *ccb, uint32_t residue, * recovered. We return an error to CAM and let CAM * retry the command if necessary. */ - xpt_freeze_devq(ccb->ccb_h.path, 1); + if (ccb->ccb_h.path != NULL) + xpt_freeze_devq(ccb->ccb_h.path, 1); ccb->ccb_h.status = CAM_REQ_CMP_ERR | CAM_DEV_QFRZN; xpt_done(ccb); break; -- 2.37.1 From 2526f99135b712f3b920963f4aaa787c6fa11148 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 3 Jun 2017 10:32:51 +0200 Subject: [PATCH 185/310] Import reproduce-electrobsd.sh to 2022-03-21-6ca7ba2 Obtained from: ElectroBSD --- reproduce-electrobsd.sh | 156 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100755 reproduce-electrobsd.sh diff --git a/reproduce-electrobsd.sh b/reproduce-electrobsd.sh new file mode 100755 index 000000000000..01f9d2d3aed9 --- /dev/null +++ b/reproduce-electrobsd.sh @@ -0,0 +1,156 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2016-2022 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## + +# reproduce-electrobsd.sh +# +# This script builds ElectroBSD in a jail, updates the jail +# using the built binaries and builds ElectroBSD again. +# +# The second build should thus always result with the same +# binaries, even if different ElectroBSD versions were used +# for the first build, or (not yet tested) if the first build +# was done with FreeBSD binaries. +# +# The following steps are not yet automated: +# +# 1) Create an /etc/jail.conf with a section like this one: +# +# ElectroBSD-amd64 { +# host.hostname=ElectroBSD-amd64; +# persist; +# children.max=0; +# allow.mount; +# allow.mount.devfs; +# enforce_statfs=1; +# path=/usr/jails/ElectroBSD-amd64; +# exec.start="mount -t devfs devfs /dev"; +# exec.stop="umount /dev"; +# } +# +# 2) Create three new ZFS datasets for the build jail: +# +# /usr/jails/ElectroBSD-amd64 +# /usr/jails/ElectroBSD-amd64/usr +# /usr/jails/ElectroBSD-amd64/usr/src +# +# 3) Extract a base.txz in it. +# # tar xf /usr/electrobsd-dist/base.txz -C /usr/jails/ElectroBSD-amd64/ +# +# 4) Mount an UFS file system in "/usr/jails/ElectroBSD-amd64/usr/obj" +# and allow the BUILD_USER to write to it. This step is only required +# to get reproducible images, the tarballs should be reproducible +# when using ZFS as well. +# +# Example commands: +# +# # zfs create -V 50G -s -o volmode=geom dpool/ufs-obj-amd64 +# # newfs -t /dev/zvol/dpool/ufs-obj-amd64 +# # mount -o async,noatime /dev/zvol/dpool/ufs-obj-amd64 /usr/jails/ElectroBSD-amd64/usr/obj/ +# # chown fk:fk /usr/jails/ElectroBSD-amd64/usr/obj/ +# +# 5) Create a dedicated /usr/src dataset on the host and populate it +# with src.txz +# +# 6) Install sudo and zogftw on the host. + +HOST_SRC_DIR="/usr/src" +HOST_BUILDLOG_DIR="/var/log/buildlogs" + +BUILD_JAIL="${BUILD_JAIL-ElectroBSD-amd64}" +BUILD_JAIL_ROOT="/usr/jails/${BUILD_JAIL}" +BUILD_JAIL_SRC_DIR="${BUILD_JAIL_ROOT}/usr/src" +BUILD_USER="$(id -un)" +BUILD_NICE_VALUE=20 +BUILD_CORE_COUNT="${BUILD_CORE_COUNT-2}" +BUILD_TWICE="${BUILD_TWICE-true}" +RESUME_BUILD="${RESUME_BUILD-false}" +# This is relative to the BUILD_JAIL_ROOT +BUILD_DISTFILE_DIR_PREFIX=/usr/obj/usr/src/ + +prepare_build_jail() { + local \ + src_dataset last_src_snapshot src_clone build_jail_dataset + + # Use vanilla zogftw configuration without potentionally + # existing fancy custom hooks that could slow us down. + export ZOGFTW_CONFIG_FILE='' + + zogftw snap /usr/src + + src_dataset=$(zogftw zcmd get_dataset_from_path "${HOST_SRC_DIR}") + if [ -z "${src_dataset}" ]; then + echo "Failed to get src dataset" + return 1 + fi + src_clone=$(zogftw zcmd get_dataset_from_path "${BUILD_JAIL_SRC_DIR}") + if [ -n "${src_clone}" ]; then + # The build jail already has a clone /usr/src, + # delete it as it may be stale + sudo zfs destroy "${src_clone}" || return 1 + else + build_jail_dataset=$(zogftw zcmd get_dataset_from_path "${BUILD_JAIL_ROOT}") + if [ -z "${build_jail_dataset}" ]; then + echo "Failed to get root jail dataset" + return 1 + fi + src_clone="${build_jail_dataset}/usr/src" + echo "Will create fresh clone in $src_clone" + fi + last_src_snapshot=$(zogftw zcmd get_last_snapshot "${src_dataset}") + if [ -z "${last_src_snapshot}" ]; then + echo "Failed to get src clone" + return 1 + fi + sudo zfs clone "${last_src_snapshot}" "${src_clone}" || return 1 +} + +reproduce() { + local \ + timestamp logfile build distfile_dir_prefix resume_flag + + timestamp=$(date "+%Y-%m-%d_%H:%M") + logfile="${HOST_BUILDLOG_DIR}/reprolog-${timestamp}" + build="$(grep ^BUILD= /usr/src/reproduce.conf | cut -d = -f 2)" + distfile_dir_prefix="/usr/obj/usr/src/$build" + + if ${RESUME_BUILD}; then + resume_flag="-r" + fi + + script "${logfile}" \ + sudo nice -n "${BUILD_NICE_VALUE}" \ + sh -c "jail -c '${BUILD_JAIL}' && \ + jexec -u ${BUILD_USER} '${BUILD_JAIL}' \ + /usr/src/reproduce.sh -j${BUILD_CORE_COUNT} -d ${distfile_dir_prefix}-j1 ${resume_flag} -a && \ + jexec '${BUILD_JAIL}' make -C /usr/src installworld NO_FSCHG='yes' && \ + ${BUILD_TWICE} && jexec -u ${BUILD_USER} '${BUILD_JAIL}' \ + /usr/src/reproduce.sh -j${BUILD_CORE_COUNT} -d ${distfile_dir_prefix}-j2; + jail -r '${BUILD_JAIL}'" || return 1 + + # Compare checksums. Use a separate script call to make + # sure the checksums from the second run are flushed to + # the file + script -a "${logfile}" sh -c "grep ^SHA256 '${logfile}' | sort -k 4 | column -t" +} + +main() { + prepare_build_jail || return 1 + reproduce || return 1 +} + +main -- 2.37.1 From fcaaa735aebae34f85b6e2f9161628d27253780c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 Feb 2016 16:24:43 +0100 Subject: [PATCH 186/310] sys/modules: Skip build of mlx4 and mlx5 when building without OFED Fixes: ===> mlx (depend) machine -> /usr/src/sys/amd64/include x86 -> /usr/src/sys/x86/include awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/kern/device_if.m -h awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/kern/bus_if.m -h awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/dev/pci/pci_if.m -h rm -f .depend CC='cc' mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I/usr/src/sys -I/usr/obj/usr/src/sys/ELECTRO_BLOAT -D__printf__=__freebsd_kprintf__ -std=iso9899:1999 -include /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_global.h /usr/src/sys/modules/mlx/../../dev/mlx/mlx.c /usr/src/sys/modules/mlx/../../dev/mlx/mlx_pci.c /usr/src/sys/modules/mlx/../../dev/mlx/mlx_disk.c ===> mlx5 (depend) machine -> /usr/src/sys/amd64/include x86 -> /usr/src/sys/x86/include awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/kern/device_if.m -h awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/kern/bus_if.m -h awk -f /usr/src/sys/tools/vnode_if.awk /usr/src/sys/kern/vnode_if.src -p awk -f /usr/src/sys/tools/vnode_if.awk /usr/src/sys/kern/vnode_if.src -q awk -f /usr/src/sys/tools/vnode_if.awk /usr/src/sys/kern/vnode_if.src -h awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/dev/pci/pci_if.m -h ln -sf /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_inet.h opt_inet.h ln -sf /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_inet6.h opt_inet6.h ln -sf /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_random.h opt_random.h ln -sf /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_rss.h opt_rss.h rm -f .depend CC='cc' mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -I/usr/src/sys/modules/mlx5/../../ofed/include -I/usr/src/sys/modules/mlx5/../../compat/linuxkpi/common/include -DHAVE_KERNEL_OPTION_HEADERS -I. -I /usr/src/sys -I/usr/obj/usr/src/sys/ELECTRO_BLOAT -D__printf__=__freebsd_kprintf__ -std=iso9899:1999 -include /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_global.h /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core /mlx5_alloc.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cmd.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cq.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_eq.c /usr/src/sys /modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_eswitch_vacl.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_flow_table.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_fw.c /usr/src/sys/modules/ mlx5/../../dev/mlx5/mlx5_core/mlx5_health.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_mad.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_main.c /usr/src/sys/modules/mlx5/../../dev/mlx5/ mlx5_core/mlx5_mcg.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_mr.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_pagealloc.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_pd.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_port.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_qp.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_srq.c /usr/src/sys/modules/ml x5/../../dev/mlx5/mlx5_core/mlx5_transobj.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_uar.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_vport.c /usr/src/sys/modules/mlx5/../../dev/mlx5 /mlx5_core/mlx5_wq.c In file included from /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_alloc.c:33: In file included from /usr/src/sys/dev/mlx5/driver.h:41: /usr/src/sys/dev/mlx5/device.h:32:10: fatal error: 'rdma/ib_verbs.h' file not found ^ 1 error generated. In file included from /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cmd.c:38: In file included from /usr/src/sys/dev/mlx5/driver.h:41: /usr/src/sys/dev/mlx5/device.h:32:10: fatal error: 'rdma/ib_verbs.h' file not found ^ 1 error generated. In file included from /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cq.c:31: In file included from /usr/src/sys/dev/mlx5/driver.h:41: /usr/src/sys/dev/mlx5/device.h:32:10: fatal error: 'rdma/ib_verbs.h' file not found ^ 1 error generated. [...] and: cc -target x86_64-unknown-freebsd11.1 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -I/usr/src/sys/ofed/include -I/usr/src/sys/compat/ linuxkpi/common/include -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_global.h -I. -I/usr/src/sys -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/obj/usr/src/sys/ELEC TRO_BLOAT -MD -MF.depend.mlx4_catas.o -MTmlx4_catas.o -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -gdwarf-2 -Wall -Wredundant-decl s -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragma s -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno-error-shift-negative-value -Wno-error-address-of-packed-member -mno-aes -mno-av x -std=iso9899:1999 -Wno-cast-qual -Wno-pointer-arith -c /usr/src/sys/dev/mlx4/mlx4_core/mlx4_catas.c -o mlx4_catas.o ctfconvert -L VERSION -g mlx4_catas.o cc -target x86_64-unknown-freebsd11.1 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -I/usr/src/sys/ofed/include -I/usr/src/sys/compat/ linuxkpi/common/include -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_global.h -I. -I/usr/src/sys -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/obj/usr/src/sys/ELEC TRO_BLOAT -MD -MF.depend.mlx4_cmd.o -MTmlx4_cmd.o -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -gdwarf-2 -Wall -Wredundant-decls -$ nested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -$ no-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno-error-shift-negative-value -Wno-error-address-of-packed-member -mno-aes -mno-avx $ std=iso9899:1999 -Wno-cast-qual -Wno-pointer-arith -c /usr/src/sys/dev/mlx4/mlx4_core/mlx4_cmd.c -o mlx4_cmd.o /usr/src/sys/dev/mlx4/mlx4_core/mlx4_cmd.c:45:10: fatal error: 'rdma/ib_smi.h' file not found #include ^~~~~~~~~~~~~~~ 1 error generated. *** Error code 1 Stop. make[4]: stopped in /usr/src/sys/modules/mlx4 *** Error code 1 Stop. make[3]: stopped in /usr/src/sys/modules *** Error code 1 Stop. make[2]: stopped in /usr/obj/usr/src/sys/ELECTRO_BLOAT *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src Obtained from: ElectroBSD --- sys/modules/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index ffd0cf9b0427..94e73b8f9530 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -531,6 +531,7 @@ _iwlwifi= iwlwifi .if ${MK_SOURCELESS_UCODE} != "no" _iwlwififw= iwlwififw .endif +.if ${MK_OFED} != "no" || defined(ALL_MODULES) _mlx4= mlx4 _mlx5= mlx5 .if (${MK_INET_SUPPORT} != "no" && ${MK_INET6_SUPPORT} != "no") || \ @@ -538,6 +539,7 @@ _mlx5= mlx5 _mlx4en= mlx4en _mlx5en= mlx5en .endif +.endif .if ${MK_OFED} != "no" || defined(ALL_MODULES) _mthca= mthca _mlx4ib= mlx4ib -- 2.37.1 From 3785cc6fd86a5c924812db9ba4c9ada4c74834b4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 6 Apr 2018 17:51:04 +0200 Subject: [PATCH 187/310] Regenerate src.conf.5 (XXX: somewhat bogus) The content is somewhat bogus and contains duplicated content but keeping the upstream version makes even less sense as many defaults are different. File generated with: cd /usr/src/tools/build/options && sh makeman > ../../../share/man/man5/src.conf.5 Obtained from: ElectroBSD --- share/man/man5/src.conf.5 | 263 ++++++++++---------------------------- 1 file changed, 67 insertions(+), 196 deletions(-) diff --git a/share/man/man5/src.conf.5 b/share/man/man5/src.conf.5 index 272ad1cb9e71..aee67a845c1e 100644 --- a/share/man/man5/src.conf.5 +++ b/share/man/man5/src.conf.5 @@ -1,6 +1,6 @@ .\" DO NOT EDIT-- this file is @generated by tools/build/options/makeman. .\" $FreeBSD$ -.Dd July 18, 2022 +.Dd August 1, 2022 .Dt SRC.CONF 5 .Os .Sh NAME @@ -101,6 +101,7 @@ Do not build .Xr acpiconf 8 , .Xr acpidump 8 and related programs. +.It Va WITH_AMD .It Va WITHOUT_APM Do not build .Xr apm 8 , @@ -114,7 +115,7 @@ checks. Do not build .Xr at 1 and related utilities. -.It Va WITHOUT_ATM +.It Va WITH_ATM Do not build programs and libraries related to ATM networking. .It Va WITHOUT_AUDIT @@ -204,7 +205,7 @@ support, like .Xr ftpd 8 , and .Xr sshd 8 . -.It Va WITHOUT_BLUETOOTH +.It Va WITH_BLUETOOTH Do not build Bluetooth related kernel modules, programs and libraries. .It Va WITHOUT_BOOT Do not build the boot blocks and loader. @@ -214,7 +215,7 @@ Do not build or install .It Va WITHOUT_BOOTPD Do not build or install .Xr bootpd 8 . -.It Va WITHOUT_BSDINSTALL +.It Va WITH_BSDINSTALL Do not build .Xr bsdinstall 8 , .Xr sade 8 , @@ -222,6 +223,7 @@ and related programs. .It Va WITHOUT_BSD_CPIO Do not build the BSD licensed version of cpio based on .Xr libarchive 3 . +.It Va WITHOUT_BSD_GREP_FASTMATCH .It Va WITHOUT_BSNMP Do not build or install .Xr bsnmpd 1 @@ -296,7 +298,7 @@ CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' See .Xr ccache 1 for more configuration options. -.It Va WITHOUT_CCD +.It Va WITH_CCD Do not build .Xr geom_ccd 4 and related utilities. @@ -331,7 +333,7 @@ Do not build the Clang C/C++ compiler during the bootstrap phase of the build. To be able to build the system, either gcc or clang bootstrap must be enabled unless an alternate compiler is provided via XCC. -.It Va WITH_CLANG_EXTRAS +.It Va WITHOUT_CLANG_EXTRAS Build additional clang and llvm tools, such as bugpoint and clang-format. .It Va WITH_CLANG_FORMAT @@ -418,20 +420,12 @@ is set explicitly) Compile with CTF (Compact C Type Format) data. CTF data encapsulates a reduced form of debugging information similar to DWARF and the venerable stabs and is required for DTrace. +.It Va WITH_CTM .It Va WITHOUT_CUSE Do not build CUSE-related programs and libraries. .It Va WITHOUT_CXGBETOOL Do not build .Xr cxgbetool 8 -.Pp -This is a default setting on -arm/armv6, arm/armv7, mips/mips, mips/mips64, powerpc/powerpc, riscv/riscv64 and riscv/riscv64sf. -.It Va WITH_CXGBETOOL -Build -.Xr cxgbetool 8 -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386 and powerpc/powerpc64. .It Va WITHOUT_CXX Do not build C++ headers and runtime libraries. It also prevents building binaries and libraries written in C++, including @@ -590,17 +584,6 @@ Set not to build .Xr efivar 3 and .Xr efivar 8 . -.Pp -This is a default setting on -mips/mips, mips/mips64, powerpc/powerpc and powerpc/powerpc64. -.It Va WITH_EFI -Build -.Xr efivar 3 -and -.Xr efivar 8 . -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_ELFTOOLCHAIN_BOOTSTRAP Do not build ELF Tool Chain tools (addr2line, nm, size, strings and strip) @@ -608,7 +591,7 @@ as part of the bootstrap process. .Bf -symbolic An alternate bootstrap tool chain must be provided. .Ef -.It Va WITHOUT_EXAMPLES +.It Va WITH_EXAMPLES Avoid installing examples to .Pa /usr/share/examples/ . .It Va WITH_EXPERIMENTAL @@ -633,11 +616,11 @@ Do not build or install .Xr finger 1 and .Xr fingerd 8 . -.It Va WITHOUT_FLOPPY +.It Va WITH_FLOPPY Do not build or install programs for operating floppy disk driver. -.It Va WITHOUT_FMTREE -Set to not build and install +.It Va WITH_FMTREE +Set to build and install .Pa /usr/sbin/fmtree . .It Va WITHOUT_FORMAT_EXTENSIONS Do not enable @@ -650,16 +633,20 @@ Build bootloaders without Forth support. Build .Nm libc without floating-point support. -.It Va WITHOUT_FREEBSD_UPDATE +.It Va WITH_FREEBSD_UPDATE Do not build .Xr freebsd-update 8 . -.It Va WITHOUT_FTP +.It Va WITH_FTP Do not build or install .Xr ftp 1 and .Xr ftpd 8 . .It Va WITHOUT_GAMES Do not build games. +.It Va WITH_GCOV +Build and install the GNU +.Xr gcov 1 +tool. .It Va WITHOUT_GH_BC Install the traditional FreeBSD .Xr bc 1 @@ -674,17 +661,6 @@ Neither build nor install .Lb libgmock , .Lb libgtest , and dependent tests. -.Pp -This is a default setting on -mips/mips and mips/mips64. -.It Va WITH_GOOGLETEST -Build and install -.Lb libgmock , -.Lb libgtest , -and dependent tests. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, powerpc/powerpc, powerpc/powerpc64, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_GPIO Do not build .Xr gpioctl 8 @@ -701,14 +677,6 @@ Build Hesiod support. Do not build HTML docs. .It Va WITHOUT_HYPERV Do not build or install HyperV utilities. -.Pp -This is a default setting on -arm/armv6, arm/armv7, arm64/aarch64, mips/mips, mips/mips64, powerpc/powerpc, powerpc/powerpc64, riscv/riscv64 and riscv/riscv64sf. -.It Va WITH_HYPERV -Build or install HyperV utilities. -.Pp -This is a default setting on -amd64/amd64 and i386/i386. .It Va WITHOUT_ICONV Do not build iconv as part of libc. .It Va WITHOUT_INCLUDES @@ -768,11 +736,11 @@ command. The user still must set the .Va DESTDIR variable to point to a directory where the user has write permissions. -.It Va WITHOUT_IPFILTER +.It Va WITH_IPFILTER Do not build IP Filter package. .It Va WITHOUT_IPFW Do not build IPFW tools. -.It Va WITHOUT_IPSEC_SUPPORT +.It Va WITH_IPSEC_SUPPORT Do not build the kernel with .Xr ipsec 4 support. @@ -780,7 +748,7 @@ This option is needed for .Xr ipsec 4 and .Xr tcpmd5 4 . -.It Va WITHOUT_ISCSI +.It Va WITH_ISCSI Do not build .Xr iscsid 8 and related utilities. @@ -864,7 +832,14 @@ On 64-bit platforms, do not build 32-bit library set and a runtime linker. .Pp This is a default setting on -arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mips, powerpc/powerpc, riscv/riscv64 and riscv/riscv64sf. +i386/i386. +.It Va WITH_LIB32 +On 64-bit platforms, do not build 32-bit library set and a +.Nm ld-elf32.so.1 +runtime linker. +.Pp +This is a default setting on +amd64/amd64. .It Va WITHOUT_LIBCPLUSPLUS Set to avoid building libcxxrt and libc++. .It Va WITH_LIBSOFT @@ -874,14 +849,6 @@ This option is for transitioning to the new hard float ABI. Do not build LLVM's lld linker. .It Va WITHOUT_LLDB Do not build the LLDB debugger. -.Pp -This is a default setting on -arm/armv6, arm/armv7, mips/mips, mips/mips64, riscv/riscv64 and riscv/riscv64sf. -.It Va WITH_LLDB -Build the LLDB debugger. -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386, powerpc/powerpc and powerpc/powerpc64. .It Va WITHOUT_LLD_BOOTSTRAP Do not build the LLD linker during the bootstrap phase of the build. @@ -974,44 +941,12 @@ Enable firewire support in /boot/loader on x86. This option is a nop on all other platforms. .It Va WITHOUT_LOADER_GELI Disable inclusion of GELI crypto support in the boot chain binaries. -.Pp -This is a default setting on -powerpc/powerpc and powerpc/powerpc64. -.It Va WITH_LOADER_GELI -Build GELI bootloader support. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mips, mips/mips64, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_LOADER_LUA Do not build LUA bindings for the boot loader. -.Pp -This is a default setting on -powerpc/powerpc and powerpc/powerpc64. -.It Va WITH_LOADER_LUA -Build LUA bindings for the boot loader. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mips, mips/mips64, riscv/riscv64 and riscv/riscv64sf. -.It Va WITHOUT_LOADER_OFW -Disable building of openfirmware bootloader components. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mips, mips/mips64, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_LOADER_OFW Build openfirmware bootloader components. -.Pp -This is a default setting on -powerpc/powerpc and powerpc/powerpc64. -.It Va WITHOUT_LOADER_UBOOT -Disable building of ubldr. -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_LOADER_UBOOT Build ubldr. -.Pp -This is a default setting on -arm/armv6, arm/armv7, mips/mips, mips/mips64, powerpc/powerpc and powerpc/powerpc64. .It Va WITH_LOADER_VERBOSE Build with extra verbose debugging in the loader. May explode already nearly too large loader over the limit. @@ -1043,6 +978,7 @@ The kernel has to be built with a module to parse the manifest. .Pp Depends on .Va WITH_LOADER_VERIEXEC . +.It Va WITH_LOADER_VERIEXEC_VECTX .It Va WITHOUT_LOADER_ZFS Do not build ZFS file system boot loader support. .It Va WITHOUT_LOCALES @@ -1107,6 +1043,7 @@ is set explicitly) .It Va WITHOUT_MANCOMPRESS Do not install compressed man pages. Only the uncompressed versions will be installed. +.It Va WITHOUT_MANDOCDB .It Va WITH_MANSPLITPKG Split man pages into their own packages during make package. .It Va WITHOUT_MAN_UTILS @@ -1173,46 +1110,30 @@ This must be set in the environment, make command line, or .Pa /etc/src-env.conf , not .Pa /etc/src.conf . -.It Va WITHOUT_MLX5TOOL -Do not build -.Xr mlx5tool 8 -.Pp -This is a default setting on -arm/armv6, arm/armv7, mips/mips, mips/mips64, powerpc/powerpc, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_MLX5TOOL Build .Xr mlx5tool 8 -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386 and powerpc/powerpc64. -.It Va WITHOUT_NDIS -Set to not build programs and libraries +.It Va WITH_NAND +Build the NAND Flash components. +.It Va WITH_NDIS +Set to build programs and libraries related to NDIS emulation support. .It Va WITHOUT_NETCAT Do not build .Xr nc 1 utility. -.It Va WITHOUT_NETGRAPH +.It Va WITH_NETGRAPH Do not build applications to support .Xr netgraph 4 . -When set, it enforces these options: -.Pp -.Bl -item -compact -.It -.Va WITHOUT_ATM -.It -.Va WITHOUT_BLUETOOTH -.El -.Pp When set, these options are also in effect: .Pp .Bl -inset -compact -.It Va WITHOUT_NETGRAPH_SUPPORT +.It Va WITH_NETGRAPH_SUPPORT (unless -.Va WITH_NETGRAPH_SUPPORT +.Va WITHOUT_NETGRAPH_SUPPORT is set explicitly) .El -.It Va WITHOUT_NETGRAPH_SUPPORT +.It Va WITH_NETGRAPH_SUPPORT Build libraries, programs, and kernel modules without netgraph support. .It Va WITHOUT_NIS Do not build @@ -1247,24 +1168,10 @@ Do not build and related programs. .It Va WITHOUT_NVME Do not build nvme related tools and kernel modules. -.Pp -This is a default setting on -arm/armv6, arm/armv7, mips/mips, mips/mips64, powerpc/powerpc, riscv/riscv64 and riscv/riscv64sf. -.It Va WITH_NVME -Build nvme related tools and kernel modules. -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386 and powerpc/powerpc64. -.It Va WITHOUT_OFED +.It Va WITH_OFED Disable the build of the .Dq "OpenFabrics Enterprise Distribution" Infiniband software stack, including kernel modules and userspace libraries. -When set, it enforces these options: -.Pp -.Bl -item -compact -.It -.Va WITHOUT_OFED_EXTRA -.El .It Va WITH_OFED_EXTRA Build the non-essential components of the .Dq "OpenFabrics Enterprise Distribution" @@ -1273,14 +1180,6 @@ Infiniband software stack, mostly examples. Enable building LDAP support for kerberos using an openldap client from ports. .It Va WITHOUT_OPENMP Do not build LLVM's OpenMP runtime. -.Pp -This is a default setting on -arm/armv6, arm/armv7, mips/mips, mips/mips64 and powerpc/powerpc. -.It Va WITH_OPENMP -Build LLVM's OpenMP runtime. -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386, powerpc/powerpc64, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_OPENSSH Do not build OpenSSH. .It Va WITHOUT_OPENSSL @@ -1326,12 +1225,12 @@ is set explicitly) Do not include kernel TLS support in OpenSSL. .Pp This is a default setting on -arm/armv6, arm/armv7, i386/i386, mips/mips, mips/mips64, powerpc/powerpc, powerpc/powerpc64, riscv/riscv64 and riscv/riscv64sf. +i386/i386. .It Va WITH_OPENSSL_KTLS Include kernel TLS support in OpenSSL. .Pp This is a default setting on -amd64/amd64 and arm64/aarch64. +amd64/amd64. .It Va WITHOUT_PAM Do not build PAM library and modules. .Bf -symbolic @@ -1350,6 +1249,7 @@ Build some programs without PAM support, particularly .Xr ftpd 8 and .Xr ppp 8 . +.It Va WITH_PC_SYSINSTALL .It Va WITHOUT_PF Do not build PF firewall package. When set, it enforces these options: @@ -1365,32 +1265,21 @@ Position-Independent Executable (PIE). Do not build .Xr pkg 7 bootstrap tool. -.It Va WITHOUT_PMC +.It Va WITH_PMC Do not build .Xr pmccontrol 8 and related programs. -.It Va WITHOUT_PORTSNAP +.It Va WITH_PORTSNAP Do not build or install .Xr portsnap 8 and related files. -.It Va WITHOUT_PPP +.It Va WITH_PPP Do not build .Xr ppp 8 and related programs. .It Va WITHOUT_PROFILE Do not build profiled libraries for use with .Xr gprof 8 . -.Pp -This is a default setting on -mips/mips64. -.It Va WITH_PROFILE -Build profiled libraries for use with -.Xr gprof 8 . -This option is deprecated and may not be present in a future version of -.Fx . -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mips, powerpc/powerpc, powerpc/powerpc64, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_QUOTAS Do not build .Xr quota 1 @@ -1412,39 +1301,35 @@ and support in .Xr ifconfig 8 , by proxy. -.It Va WITHOUT_RBOOTD +.It Va WITH_RBOOTD Do not build or install .Xr rbootd 8 . -.It Va WITH_REPRODUCIBLE_BUILD -Exclude build metadata (such as the build time, user, or host) -from the kernel, boot loaders, and uname output, so that builds produce -bit-for-bit identical output. +.It Va WITH_RCMDS +.It Va WITHOUT_RCS +.It Va WITHOUT_REPRODUCIBLE_BUILD +Include build metadata (such as the build time, user, and host) +in the kernel, boot loaders, and uname output. +Successive builds will not be bit-for-bit identical. .It Va WITHOUT_RESCUE Do not build .Xr rescue 8 . .It Va WITH_RETPOLINE Build the base system with the retpoline speculative execution vulnerability mitigation for CVE-2017-5715. -.It Va WITHOUT_ROUTED -Do not build -.Xr routed 8 -utility. .It Va WITH_RPCBIND_WARMSTART_SUPPORT Build .Xr rpcbind 8 with warmstart support. -.It Va WITHOUT_SENDMAIL -Do not build +.It Va WITH_SCTP_SUPPORT +.It Va WITH_SENDMAIL +Build and install .Xr sendmail 8 and related programs. -.It Va WITHOUT_SERVICESDB -Do not install -.Pa /var/db/services.db . .It Va WITHOUT_SETUID_LOGIN Set this to disable the installation of .Xr login 1 as a set-user-ID root program. -.It Va WITHOUT_SHAREDOCS +.It Va WITH_SHAREDOCS Do not build the .Bx 4.4 legacy docs. @@ -1457,19 +1342,11 @@ and necessary utilities like assembler, linker and library archive manager. .It Va WITH_SORT_THREADS Enable threads in .Xr sort 1 . -.It Va WITHOUT_SOURCELESS +.It Va WITH_SOURCELESS Do not build kernel modules that include sourceless code (either microcode or native code for host CPU). -When set, it enforces these options: -.Pp -.Bl -item -compact -.It -.Va WITHOUT_SOURCELESS_HOST -.It -.Va WITHOUT_SOURCELESS_UCODE -.El -.It Va WITHOUT_SOURCELESS_HOST +.It Va WITH_SOURCELESS_HOST Do not build kernel modules that include sourceless native code for host CPU. -.It Va WITHOUT_SOURCELESS_UCODE +.It Va WITH_SOURCELESS_UCODE Do not build kernel modules that include sourceless microcode. .It Va WITHOUT_SPLIT_KERNEL_DEBUG Do not build standalone kernel debug files. @@ -1483,14 +1360,6 @@ When set, it enforces these options: .El .It Va WITHOUT_SSP Do not build world with propolice stack smashing protection. -.Pp -This is a default setting on -mips/mips and mips/mips64. -.It Va WITH_SSP -Build world with propolice stack smashing protection. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, powerpc/powerpc, powerpc/powerpc64, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_STAGING Enable staging of files to a stage tree. This can be best thought of as auto-install to @@ -1538,6 +1407,7 @@ and related programs. Do not build .Xr syscons 4 support files such as keyboard maps, fonts, and screen output maps. +.It Va WITH_SYSINSTALL .It Va WITH_SYSROOT Enable use of sysroot during build. Depends on @@ -1576,11 +1446,11 @@ Do not build or install .Xr talk 1 and .Xr talkd 8 . -.It Va WITHOUT_TCP_WRAPPERS -Do not build or install +.It Va WITH_TCP_WRAPPERS +Build and install .Xr tcpd 8 , and related utilities. -.It Va WITHOUT_TCSH +.It Va WITH_TCSH Do not build and install .Pa /bin/csh (which is @@ -1628,11 +1498,12 @@ When set, it enforces these options: .It Va WITHOUT_TEXTPROC Do not build programs used for text processing. -.It Va WITHOUT_TFTP +.It Va WITH_TFTP Do not build or install .Xr tftp 1 and .Xr tftpd 8 . +.It Va WITH_TIMED .It Va WITHOUT_TOOLCHAIN Do not install header or programs used for program development, -- 2.37.1 From 0067c7fb15de95881f0a8fde1ce07e4fa2d386dc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 16 Aug 2017 11:10:00 +0200 Subject: [PATCH 188/310] cam iosched: Fix trim statistics (XXX: may not be correct for nvme) When cam_iosched_bio_complete() gets called, all pending BIO_DELETEs are done. Previously only one of them was accounted for, as a result the "pending" counter got higher and higher and the "out" count was off. PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221952 PR submission date: 2017-08-31 Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 50b6db88cebd..74dfa0522418 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -1676,8 +1676,15 @@ cam_iosched_bio_complete(struct cam_iosched_softc *isc, struct bio *bp, } else if (bp->bio_cmd == BIO_DELETE) { if ((bp->bio_flags & BIO_ERROR) != 0) isc->trim_stats.errs++; - isc->trim_stats.out++; - isc->trim_stats.pending--; + /* + * A single request sent to the disk may trim multiple + * areas that belong to different BIO_DELETE requests. + * We only sent one trim request to the disk at the time, + * when it completes all pending BIO_DELETEs are done and + * have to be counted as such. + */ + isc->trim_stats.out += isc->trim_stats.pending; + isc->trim_stats.pending = 0; } else if (bp->bio_cmd != BIO_FLUSH) { if (iosched_debug) printf("Completing command with bio_cmd == %#x\n", bp->bio_cmd); -- 2.37.1 From 707cc189f3960f3cebc377b6dad65796838689b3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 22 Jul 2017 20:12:40 +0200 Subject: [PATCH 189/310] cam_iosched: Add a sysctl for the iosched debug level The global iosched sysctls should probably get their own sysctl subtree ... Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 74dfa0522418..a8c3bd180032 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -104,6 +104,9 @@ struct iop_stats; struct cam_iosched_softc; int iosched_debug = 0; +TUNABLE_INT("kern.cam.iosched_debug", &iosched_debug); +SYSCTL_INT(_kern_cam, OID_AUTO, iosched_debug, CTLFLAG_RW, + &iosched_debug, 1, "Dynamic I/O scheduler debug level"); typedef enum { none = 0, /* No limits */ -- 2.37.1 From 621fe1b03b7cfdfb5609d12921bcdb2d0f8f9cbd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 25 Jul 2017 13:06:48 +0200 Subject: [PATCH 190/310] cam iosched: Optionally only enforce write limits if there are other requests pending ... or reads queued. This reduces write latency for many workloads without impacting read latency (too much). An example workload that profits a lot from this is watching a DVD rip from an USB disk while adding DVD rips using rsync. In this case the writes only have to be throttled when VLC is reading which only happens every couple of seconds. Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index a8c3bd180032..97bd44a925e3 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -286,6 +286,7 @@ struct cam_iosched_softc { int current_read_bias; /* Current read bias state */ int total_ticks; int load; /* EMA of 'load average' of disk / 2^16 */ + int ignore_write_limiter_if_no_ops_are_pending; struct bio_queue_head write_queue; struct iop_stats read_stats, write_stats, trim_stats; @@ -738,6 +739,10 @@ cam_iosched_has_io(struct cam_iosched_softc *isc) cam_iosched_limiter_caniop(&isc->write_stats, wbp) == 0; bool can_read = rbp != NULL && cam_iosched_limiter_caniop(&isc->read_stats, rbp) == 0; + if (isc->ignore_write_limiter_if_no_ops_are_pending && + !can_read && wbp != NULL && !can_write) { + can_write = 1; + } if (iosched_debug > 2) { printf("can write %d: pending_writes %d max_writes %d\n", can_write, isc->write_stats.pending, isc->write_stats.max); printf("can read %d: read_stats.pending %d max_reads %d\n", can_read, isc->read_stats.pending, isc->read_stats.max); @@ -1117,6 +1122,7 @@ cam_iosched_init(struct cam_iosched_softc **iscp, struct cam_periph *periph) (*iscp)->read_bias = 100; (*iscp)->current_read_bias = 100; (*iscp)->quanta = min(hz, 200); + (*iscp)->ignore_write_limiter_if_no_ops_are_pending = 0; cam_iosched_iop_stats_init(*iscp, &(*iscp)->read_stats); cam_iosched_iop_stats_init(*iscp, &(*iscp)->write_stats); cam_iosched_iop_stats_init(*iscp, &(*iscp)->trim_stats); @@ -1202,6 +1208,11 @@ void cam_iosched_sysctl_init(struct cam_iosched_softc *isc, &isc->read_bias, 100, "How biased towards read should we be independent of limits"); + SYSCTL_ADD_INT(ctx, n, + OID_AUTO, "ignore_write_limiter_if_no_ops_are_pending", CTLFLAG_RW, + &isc->ignore_write_limiter_if_no_ops_are_pending, 0, + "Only enforce write limits when there are ops pending"); + SYSCTL_ADD_PROC(ctx, n, OID_AUTO, "quanta", CTLTYPE_UINT | CTLFLAG_RW | CTLFLAG_MPSAFE, &isc->quanta, 0, cam_iosched_quanta_sysctl, "I", @@ -1327,10 +1338,20 @@ cam_iosched_get_write(struct cam_iosched_softc *isc) * See if our current limiter allows this I/O. */ if (cam_iosched_limiter_iop(&isc->write_stats, bp) != 0) { - if (iosched_debug) - printf("Can't write because limiter says no.\n"); - isc->write_stats.state_flags |= IOP_RATE_LIMITED; - return NULL; + if (isc->ignore_write_limiter_if_no_ops_are_pending && + !bioq_first(&isc->bio_queue) && + isc->read_stats.pending == 0 && + isc->write_stats.pending == 0 && + isc->trim_stats.pending == 0) { + if (iosched_debug) + printf("Write limiter says no but no ops " + "are pending. Allowing write.\n"); + } else { + if (iosched_debug) + printf("Can't write because limiter says no.\n"); + isc->write_stats.state_flags |= IOP_RATE_LIMITED; + return NULL; + } } /* -- 2.37.1 From 010fa37021337ebe851f03580fbeb50ed2c86301 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 21 Jul 2017 16:11:21 +0200 Subject: [PATCH 191/310] cam iosched: Include the valid values in the cam_iosched_limiter_sysctl description Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 97bd44a925e3..15c2ee312830 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -1019,7 +1019,7 @@ cam_iosched_iop_stats_sysctl_init(struct cam_iosched_softc *isc, struct iop_stat OID_AUTO, "limiter", CTLTYPE_STRING | CTLFLAG_RW | CTLFLAG_MPSAFE, ios, 0, cam_iosched_limiter_sysctl, "A", - "Current limiting type."); + "Current limiting type. Valid values: 'none', 'queue_depth', 'iops' and 'bandwidth'"); SYSCTL_ADD_INT(ctx, n, OID_AUTO, "min", CTLFLAG_RW, &ios->min, 0, -- 2.37.1 From 03fee4ccfc9a8054c4f505488ab460110bca19a7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 2 Aug 2017 11:53:23 +0200 Subject: [PATCH 192/310] cam iosched: Include the accepted control type algorithms in the sysctl description ... so the user doesn't have to refer to the source code or the (non-existant) man page. Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 15c2ee312830..1b70971c0a2e 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -1067,7 +1067,7 @@ cam_iosched_cl_sysctl_init(struct cam_iosched_softc *isc) OID_AUTO, "type", CTLTYPE_STRING | CTLFLAG_RW | CTLFLAG_MPSAFE, clp, 0, cam_iosched_control_type_sysctl, "A", - "Control loop algorithm"); + "Control loop algorithm. Can be 'set_max' or 'read_latency'."); SYSCTL_ADD_PROC(ctx, n, OID_AUTO, "steer_interval", CTLTYPE_STRING | CTLFLAG_RW | CTLFLAG_MPSAFE, -- 2.37.1 From 9ecd5cf5bd7cb0c9103982eb5570145fc1cc112b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 8 Aug 2017 09:10:27 +0200 Subject: [PATCH 193/310] cam iosched: Only log the 'Steering write' message if writes are actually limited Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 1b70971c0a2e..fa333b31e061 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -642,7 +642,8 @@ cam_iosched_cl_maybe_steer(struct control_loop *clp) clp->next_steer = now + clp->steer_interval; switch (clp->type) { case set_max: - if (isc->write_stats.current != isc->write_stats.max) + if (isc->write_stats.current != isc->write_stats.max && + isc->write_stats.limiter != none) { printf("Steering write from %d kBps to %d kBps\n", isc->write_stats.current, isc->write_stats.max); isc->read_stats.current = isc->read_stats.max; -- 2.37.1 From 831b2674098cdcd080ae55f8804457f2dad46dba Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 3 Aug 2017 10:42:23 +0200 Subject: [PATCH 194/310] cam_iosched: Use the correct units when logging write limit changes ... for the 'queue_depth' or 'iops' limiter. Previously "kBps" were used unconditionally. Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index fa333b31e061..58cb5d77aeda 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -119,6 +119,9 @@ typedef enum { static const char *cam_iosched_limiter_names[] = { "none", "queue_depth", "iops", "bandwidth" }; +static char *cam_iosched_limiter_units[] = + { "none", "queued ios", "iops", "kBps" }; + /* * Called to initialize the bits of the iop_stats structure relevant to the * limiter. Called just after the limiter is set. @@ -627,6 +630,14 @@ cam_iosched_cl_init(struct control_loop *clp, struct cam_iosched_softc *isc) clp->type = set_max; } +static const char * +cam_iosched_get_limiter_unit_string(enum io_limiter limiter) +{ + + KASSERT(none <= limiter && limiter < limiter_max, "Invalid limiter"); + return cam_iosched_limiter_units[limiter]; +} + static void cam_iosched_cl_maybe_steer(struct control_loop *clp) { @@ -644,8 +655,11 @@ cam_iosched_cl_maybe_steer(struct control_loop *clp) case set_max: if (isc->write_stats.current != isc->write_stats.max && isc->write_stats.limiter != none) { - printf("Steering write from %d kBps to %d kBps\n", - isc->write_stats.current, isc->write_stats.max); + const char *units; + units = cam_iosched_get_limiter_unit_string(isc->write_stats.limiter); + printf("Steering write from %d %s to %d %s\n", + isc->write_stats.current, units, isc->write_stats.max, units); + } isc->read_stats.current = isc->read_stats.max; isc->write_stats.current = isc->write_stats.max; isc->trim_stats.current = isc->trim_stats.max; -- 2.37.1 From fa177f6a519789905e9a51d7ac728f21e53b5733 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 6 Aug 2017 19:01:53 +0200 Subject: [PATCH 195/310] cam iosched: Add sysctl to show the units used by the currently active limiter (XXX: or not) XXX: Doesn't work properly yet, sysctl always shows none. Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 58cb5d77aeda..b6eaeff7d598 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -211,6 +211,7 @@ struct iop_stats { * Information about the current rate limiters, if any */ io_limiter limiter; /* How are I/Os being limited */ + char *units; /* Current rate limiter unit (exported as sysctl) */ int min; /* Low range of limit */ int max; /* High range of limit */ int current; /* Current rate limiter */ @@ -630,8 +631,8 @@ cam_iosched_cl_init(struct control_loop *clp, struct cam_iosched_softc *isc) clp->type = set_max; } -static const char * -cam_iosched_get_limiter_unit_string(enum io_limiter limiter) +static char * +cam_iosched_get_limiter_unit_string(int limiter) { KASSERT(none <= limiter && limiter < limiter_max, "Invalid limiter"); @@ -655,10 +656,9 @@ cam_iosched_cl_maybe_steer(struct control_loop *clp) case set_max: if (isc->write_stats.current != isc->write_stats.max && isc->write_stats.limiter != none) { - const char *units; - units = cam_iosched_get_limiter_unit_string(isc->write_stats.limiter); printf("Steering write from %d %s to %d %s\n", - isc->write_stats.current, units, isc->write_stats.max, units); + isc->write_stats.current, isc->write_stats.units, + isc->write_stats.max, isc->write_stats.units); } isc->read_stats.current = isc->read_stats.max; isc->write_stats.current = isc->write_stats.max; @@ -829,6 +829,7 @@ cam_iosched_iop_stats_init(struct cam_iosched_softc *isc, struct iop_stats *ios) { ios->limiter = none; + ios->units = cam_iosched_get_limiter_unit_string(ios->limiter); ios->in = 0; ios->max = ios->current = 300000; ios->min = 1; @@ -871,9 +872,15 @@ cam_iosched_limiter_sysctl(SYSCTL_HANDLER_ARGS) if (strcmp(buf, cam_iosched_limiter_names[i]) != 0) continue; ios->limiter = i; + ios->units = cam_iosched_get_limiter_unit_string(i); error = cam_iosched_limiter_init(ios); if (error != 0) { + /* + * Continue with the previous limiter, + * ios->units remains valid. + */ ios->limiter = value; + ios->units = cam_iosched_get_limiter_unit_string(value); cam_periph_unlock(isc->periph); return error; } @@ -1035,6 +1042,10 @@ cam_iosched_iop_stats_sysctl_init(struct cam_iosched_softc *isc, struct iop_stat CTLTYPE_STRING | CTLFLAG_RW | CTLFLAG_MPSAFE, ios, 0, cam_iosched_limiter_sysctl, "A", "Current limiting type. Valid values: 'none', 'queue_depth', 'iops' and 'bandwidth'"); + SYSCTL_ADD_STRING(ctx, n, + OID_AUTO, "units", CTLFLAG_RD, + ios->units, 0, + "Units used by currently active limiter (read-only)"); SYSCTL_ADD_INT(ctx, n, OID_AUTO, "min", CTLFLAG_RW, &ios->min, 0, -- 2.37.1 From d99b0fcad92504eef3eea7290c0123bef183e3e7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 25 Jul 2017 14:05:08 +0200 Subject: [PATCH 196/310] cam_iosched: Add sysctl to flush quickly Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index b6eaeff7d598..c7322d63fde0 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -108,6 +108,11 @@ TUNABLE_INT("kern.cam.iosched_debug", &iosched_debug); SYSCTL_INT(_kern_cam, OID_AUTO, iosched_debug, CTLFLAG_RW, &iosched_debug, 1, "Dynamic I/O scheduler debug level"); +int iosched_flush_quickly = 0; +TUNABLE_INT("kern.cam.iosched_flush_quickly", &iosched_flush_quickly); +SYSCTL_INT(_kern_cam, OID_AUTO, iosched_flush_quickly, CTLFLAG_RW, + &iosched_flush_quickly, 1, "Let flushes through if they are at the beginning of the queue"); + typedef enum { none = 0, /* No limits */ queue_depth, /* Limit how many ops we queue to SIM */ @@ -363,6 +368,9 @@ cam_iosched_limiter_caniop(struct iop_stats *ios, struct bio *bp) { int lim = ios->limiter; + if (iosched_flush_quickly && bp->bio_cmd == BIO_FLUSH) { + return 0; + } /* maybe this should be a kassert */ if (lim < none || lim >= limiter_max) return EINVAL; @@ -1349,15 +1357,20 @@ cam_iosched_get_write(struct cam_iosched_softc *isc) * setting. */ if (bioq_first(&isc->bio_queue) && isc->current_read_bias) { - if (iosched_debug) - printf( - "Reads present and current_read_bias is %d queued " - "writes %d queued reads %d\n", - isc->current_read_bias, isc->write_stats.queued, - isc->read_stats.queued); - isc->current_read_bias--; - /* We're not limiting writes, per se, just doing reads first */ - return NULL; + if (iosched_flush_quickly && bp->bio_cmd == BIO_FLUSH) { + if (iosched_debug) + printf("Reads pending but op is flush. Allowing it.\n"); + } else { + if (iosched_debug) + printf( + "Reads present and current_read_bias is %d queued " + "writes %d queued reads %d\n", + isc->current_read_bias, isc->write_stats.queued, + isc->read_stats.queued); + isc->current_read_bias--; + /* We're not limiting writes, per se, just doing reads first */ + return NULL; + } } /* -- 2.37.1 From 67749dd653b5824720967e218788bc8f936c12cf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 5 Aug 2018 12:02:20 +0200 Subject: [PATCH 197/310] Unbreak the build Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index c7322d63fde0..234d8d34c25b 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -1388,7 +1388,7 @@ cam_iosched_get_write(struct cam_iosched_softc *isc) } else { if (iosched_debug) printf("Can't write because limiter says no.\n"); - isc->write_stats.state_flags |= IOP_RATE_LIMITED; + /* isc->write_stats.state_flags |= IOP_RATE_LIMITED; */ return NULL; } } -- 2.37.1 From e6ea237902aaabee0da1edda98d1ae9bb12dcdad Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 Aug 2022 10:09:12 +0200 Subject: [PATCH 198/310] share/mk/src.opts.mk: Disable NAND by default Obtained from: ElectroBSD --- share/mk/src.opts.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 19f2e46b36b7..a14a2e6289c6 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -189,6 +189,7 @@ __DEFAULT_NO_OPTIONS = \ LOADER_FIREWIRE \ LOADER_VERBOSE \ LOADER_VERIEXEC_PASS_MANIFEST \ + NAND \ OFED_EXTRA \ OPENLDAP \ RPCBIND_WARMSTART_SUPPORT \ -- 2.37.1 From a66ffe6f84d6fdeaa0252b6e0bc5eb2eafa7cbaf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 Aug 2022 10:09:36 +0200 Subject: [PATCH 199/310] share/mk/src.opts.mk: Disable OFED by default on all platforms Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index a14a2e6289c6..781fd695b045 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -140,7 +140,6 @@ __DEFAULT_YES_OPTIONS = \ NS_CACHING \ NTP \ NVME \ - OFED \ OPENSSL \ PAM \ PF \ @@ -190,6 +189,7 @@ __DEFAULT_NO_OPTIONS = \ LOADER_VERBOSE \ LOADER_VERIEXEC_PASS_MANIFEST \ NAND \ + OFED \ OFED_EXTRA \ OPENLDAP \ RPCBIND_WARMSTART_SUPPORT \ -- 2.37.1 From 5dd3545dd2b23fa6a002cbc16719b14ea8b94a88 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 11 Oct 2018 11:24:22 +0200 Subject: [PATCH 200/310] Allow to change the ostype at runtime Obtained from: ElectroBSD --- sys/kern/kern_mib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/kern_mib.c b/sys/kern/kern_mib.c index 5db99784311c..dcb9bb7fb7ff 100644 --- a/sys/kern/kern_mib.c +++ b/sys/kern/kern_mib.c @@ -105,7 +105,7 @@ SYSCTL_INT(_kern, KERN_OSREV, osrevision, CTLFLAG_RD|CTLFLAG_CAPRD, SYSCTL_STRING(_kern, KERN_VERSION, version, CTLFLAG_RD|CTLFLAG_MPSAFE, version, 0, "Kernel version"); -SYSCTL_STRING(_kern, KERN_OSTYPE, ostype, CTLFLAG_RD|CTLFLAG_MPSAFE| +SYSCTL_STRING(_kern, KERN_OSTYPE, ostype, CTLFLAG_RW|CTLFLAG_MPSAFE| CTLFLAG_CAPRD, ostype, 0, "Operating system type"); SYSCTL_INT(_kern, KERN_MAXPROC, maxproc, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, -- 2.37.1 From 8b8cbad873ffc46efa474f660d79ccb596cc67d7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 26 Nov 2018 15:06:15 +0100 Subject: [PATCH 201/310] image-checksum.sh: Adjust to deal with slices instead of GPT partitions Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 17f062200b61..00acbdcf7429 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -45,7 +45,7 @@ # ########################################################################## -UFS_PARTITION=p2 +UFS_PARTITION=s2a EXPECTED_PARTITIONS=2 MOUNTPOINT=/mnt VERBOSE=0 @@ -96,7 +96,7 @@ partition_count_acceptable() { local md_unit="${1}" # Verify that there are exactly two partitions present - partitions=$(gpart show -r -p "md${md_unit}" | grep -c "md${md_unit}"p) + partitions=$(gpart show -r -p "md${md_unit}" | grep -c "md${md_unit}"s) if [ "${partitions}" != "${EXPECTED_PARTITIONS}" ]; then echo "Invalid number of partitions: ${partitions}" return 1; @@ -148,7 +148,7 @@ generate_partial_image_checksum() { mdconfig -d -u "${md_unit}" || return 1 - bootcode_checksum=$(dd if=/dev/md${md_unit}p1 2>/dev/null | sha256) + bootcode_checksum=$(dd if=/dev/md${md_unit}s1 2>/dev/null | sha256) verbose_log "Boot code checksum: ${bootcode_checksum}" mtree_checksum=$(sha256 -q "${spec_file}") verbose_log "mtree checksum: ${mtree_checksum}" -- 2.37.1 From 53997ab6b0b0a9c7d856d3c377c88297c0848155 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 12 Feb 2020 17:32:22 +0100 Subject: [PATCH 202/310] etc/shells: Remove /bin/csh and /bin/tcsh from the list of acceptable shells Obtained from: ElectroBSD --- etc/shells | 2 -- 1 file changed, 2 deletions(-) diff --git a/etc/shells b/etc/shells index fe1e0294a010..3797f57e85f9 100644 --- a/etc/shells +++ b/etc/shells @@ -5,5 +5,3 @@ # one of these shells. /bin/sh -/bin/csh -/bin/tcsh -- 2.37.1 From e15f710f9d6251fc31945a00f7db91c04ab1ece2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 18 Feb 2020 19:26:06 +0100 Subject: [PATCH 203/310] strip-freebsd.sh: Add cp to the list of drivers to ditch (obfuscated code) Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 4b15c8e87f99..249417faa07f 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -102,6 +102,8 @@ get_tainted_sys_devs() { # use cases so figuring out the license problems is a low priority issue # because we can simply ditch the offending code. echo "tdfx" + # Intentionally obfuscated code in cpddk.c + echo "cp" } get_unused_contrib_dirs() { -- 2.37.1 From 47125cac9241e5622f7ed963c5eef0f83f58bf8d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 18 Feb 2020 19:30:51 +0100 Subject: [PATCH 204/310] strip-freebsd.sh: Add ce to the list of drivers to ditch (obfuscated code) Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 249417faa07f..c12d88416a03 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -102,7 +102,9 @@ get_tainted_sys_devs() { # use cases so figuring out the license problems is a low priority issue # because we can simply ditch the offending code. echo "tdfx" - # Intentionally obfuscated code in cpddk.c + # Intentionally obfuscated code and micro code in tau32-ddk.c + echo "ce" + # Intentionally obfuscated code and micro code in cpddk.c echo "cp" } -- 2.37.1 From 05c026fe6dd8e772f94b1f34c9d306a826b6d043 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 19 Feb 2020 13:02:01 +0100 Subject: [PATCH 205/310] release/amd64/make-memstick.sh: Use the freshly built mkimg Should prevent: Image `memstick.img.part' complete mkimg: invalid option -- a mkimg: error: unknown option usage: mkimg Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index ca8259a10354..8460ed6f0792 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -64,7 +64,7 @@ rm "${2}.mtree" espfilename=$(mktemp /tmp/efiboot.XXXXXX) make_esp_file ${espfilename} ${fat32min} ${1}/boot/loader.efi -mkimg -s mbr \ +dist/base/usr/bin/mkimg -s mbr \ -b ${1}/boot/mbr \ -p efi:=${espfilename} \ -p freebsd:-"mkimg -s bsd -b ${1}/boot/boot -p freebsd-ufs:=${2}.part" \ -- 2.37.1 From 6b76299943627fd173fd431dd2c30ff2277c8450 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 1 May 2020 18:37:44 +0200 Subject: [PATCH 206/310] Detach mt and tcopy from the build Obtained from: ElectroBSD --- usr.bin/Makefile | 2 -- 1 file changed, 2 deletions(-) diff --git a/usr.bin/Makefile b/usr.bin/Makefile index 62569bedb151..eaa87376269e 100644 --- a/usr.bin/Makefile +++ b/usr.bin/Makefile @@ -95,7 +95,6 @@ SUBDIR= alias \ mkimg \ mktemp \ mkuzip \ - mt \ ncal \ netstat \ newgrp \ @@ -149,7 +148,6 @@ SUBDIR= alias \ tabs \ tail \ tar \ - tcopy \ tee \ time \ timeout \ -- 2.37.1 From dcb93bf69ee25d64f6cf5d87ba9a91d83441e6b4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 2 May 2020 17:59:48 +0200 Subject: [PATCH 207/310] Detach gvinum from the build Obtained from: ElectroBSD --- sbin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/sbin/Makefile b/sbin/Makefile index c9b548a8a390..ece0653fdf51 100644 --- a/sbin/Makefile +++ b/sbin/Makefile @@ -27,7 +27,6 @@ SUBDIR=adjkerntz \ geom \ ggate \ growfs \ - gvinum \ ifconfig \ init \ kldconfig \ -- 2.37.1 From d715474678e5333c58651d76b121c259c452ac0b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 17 May 2020 15:20:43 +0200 Subject: [PATCH 208/310] release/rc.local: Adjust /tmp/reproduce-from-image.sh to deal with slices Obtained from: ElectroBSD --- release/rc.local | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/release/rc.local b/release/rc.local index a5977c3a8f84..877d3f4ac617 100755 --- a/release/rc.local +++ b/release/rc.local @@ -17,14 +17,14 @@ set -x -e # Fingers crossed that the first disk is the boot disk ... DISK=$(geom disk status -s | head -n 1 | cut -f 1 -w) JAILROOT="$(mktemp -d /tmp/ElectroBSD-jail-XXXXXX)" -SWAP_PARTITION_INDEX=3 -BUILD_PARTITION_INDEX=4 +SWAP_SLICE_INDEX=3 +BUILD_SLICE_INDEX=4 gpart recover "${DISK}" -gpart add -t freebsd-swap -s 1500M "${DISK}" -swapon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" -dumpon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" -gpart add -t freebsd-ufs "${DISK}" +gpart add -t freebsd -s 1500M "${DISK}" +swapon "/dev/${DISK}s${SWAP_SLICE_INDEX}" +dumpon "/dev/${DISK}s${SWAP_SLICE_INDEX}" +gpart add -t freebsd "${DISK}" # gpart partition changes made in bhyve are currently # lost. The cause hasn't been diagnosed yet. # Explicitly include the partition layout in the @@ -32,8 +32,8 @@ gpart add -t freebsd-ufs "${DISK}" # afterwards. gpart show "${DISK}" # More convenient format for humans gpart backup "${DISK}" # Same data in format understood by "gpart restore" -newfs "/dev/${DISK}p${BUILD_PARTITION_INDEX}" -mount -o async,noatime "/dev/${DISK}p${BUILD_PARTITION_INDEX}" "${JAILROOT}" +newfs "/dev/${DISK}s${BUILD_SLICE_INDEX}" +mount -o async,noatime "/dev/${DISK}s${BUILD_SLICE_INDEX}" "${JAILROOT}" tar xf /usr/electrobsd-dist/base.txz -C "${JAILROOT}" tar xf /usr/electrobsd-dist/src.txz -C "${JAILROOT}" time jail -c path="${JAILROOT}" mount.devfs host.hostname=ElectroBSD command=sh /usr/src/reproduce.sh -- 2.37.1 From e409ea22ec691aeffdbe89edc535eb600158889a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 30 May 2020 22:04:41 +0200 Subject: [PATCH 209/310] libdtrace: Fix comment typo Obtained from: ElectroBSD --- cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c index 0b3dac0224f9..6106e09e01db 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c @@ -1207,7 +1207,7 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *eprobesp) /* * We use this token as a relatively unique handle for this file on the * system in order to disambiguate potential conflicts between files of - * the same name which contain identially named local symbols. + * the same name which contain identically named local symbols. */ if ((objkey = hash_obj(obj, fd)) == (unsigned int)-1) return (dt_link_error(dtp, elf, fd, bufs, -- 2.37.1 From b2793e2dc0e4ba1a215e93acf10dfb5dc10032cb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2020 08:46:52 +0200 Subject: [PATCH 210/310] share/man/man5/fstab.5: Fix typo Obtained from: ElectroBSD --- share/man/man5/fstab.5 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/man/man5/fstab.5 b/share/man/man5/fstab.5 index a83b26288ebc..752c1c12cda1 100644 --- a/share/man/man5/fstab.5 +++ b/share/man/man5/fstab.5 @@ -426,7 +426,7 @@ resides in # Swap space on a block device. /dev/da0p1 none swap sw 0 0 # -# Swap space using a block device with GBDE/GELI encyption. +# Swap space using a block device with GBDE/GELI encryption. # aalgo, ealgo, keylen, sectorsize options are available # for .eli devices. /dev/da1p1.bde none swap sw 0 0 -- 2.37.1 From 032cde64ef2710a0ec8a074c18a8705316d32a60 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2020 15:37:36 +0200 Subject: [PATCH 211/310] Detach bsdconfig from the build Obtained from: ElectroBSD --- usr.sbin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index 43b258b0f6a0..8fbbd4d56b4d 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -125,7 +125,6 @@ SUBDIR.${MK_BSNMP}+= bsnmpd SUBDIR.${MK_OPENSSL}+= certctl .endif SUBDIR.${MK_CXGBETOOL}+= cxgbetool -SUBDIR.${MK_DIALOG}+= bsdconfig SUBDIR.${MK_EFI}+= efivar efidp efibootmgr .if ${MK_OPENSSL} != "no" SUBDIR.${MK_EFI}+= uefisign -- 2.37.1 From 1d48cc2ea21a5346a19d0ad633bd7571f74b54e8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 2 Jun 2020 12:56:14 +0200 Subject: [PATCH 212/310] strip-freebsd.sh: Ditch ice files if available They rely on a proprietary "DDP package file". Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index c12d88416a03..79d5e4bb2cb9 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -25,6 +25,17 @@ # likely to still contain lots of non-free parts that haven't # been discovered yet. +# ice relies on a proprietary "DDP package file" +# (sys/contrib/dev/ice/ice-1.3.9.0.pkg) +get_dev_ice_files() { + if [ -d sys/contrib/dev/ice ]; then + find sys/contrib/dev/ice + fi + if [ -d sys/dev/ice ]; then + find sys/dev/ice || true + fi +} + get_snd_csa_files() { find sys/dev/sound/pci -name "csa*" } @@ -56,6 +67,7 @@ get_usb_firmware_files() { # by non-free dependencies get_unfree_files() { find sys/ -name "*.uu" + get_dev_ice_files get_snd_csa_files get_snd_ds1_files get_snd_maestro3_files -- 2.37.1 From c2cd9c4e92f38f162648b0c50f9046effc5e1dc6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 5 Nov 2020 17:49:22 +0100 Subject: [PATCH 213/310] strip-freebsd.sh: Ditch qat firmware files if available Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 79d5e4bb2cb9..68fc993e33aa 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -36,6 +36,14 @@ get_dev_ice_files() { fi } +# qat firmware modules were added in r367387 and are not (yet) +# available in stable/11. +get_dev_qatfw_files() { + if [ -d sys/contrib/dev/qat ]; then + find sys/contrib/dev/qat + fi +} + get_snd_csa_files() { find sys/dev/sound/pci -name "csa*" } @@ -68,6 +76,7 @@ get_usb_firmware_files() { get_unfree_files() { find sys/ -name "*.uu" get_dev_ice_files + get_dev_qatfw_files get_snd_csa_files get_snd_ds1_files get_snd_maestro3_files -- 2.37.1 From df3f0d8496c10bd73465057e12dc7dee720fb2e1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 26 Nov 2020 11:12:11 +0100 Subject: [PATCH 214/310] strip-freebsd.sh: Ditch imx firmware if available Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 68fc993e33aa..556b2ba4e365 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -36,6 +36,14 @@ get_dev_ice_files() { fi } +# imx firmware was added in r368042 and is not (yet) +# available in stable/11. +get_dev_imx_files() { + if [ -d sys/contrib/dev/imx ]; then + find sys/contrib/dev/imx + fi +} + # qat firmware modules were added in r367387 and are not (yet) # available in stable/11. get_dev_qatfw_files() { @@ -76,6 +84,7 @@ get_usb_firmware_files() { get_unfree_files() { find sys/ -name "*.uu" get_dev_ice_files + get_dev_imx_files get_dev_qatfw_files get_snd_csa_files get_snd_ds1_files -- 2.37.1 From 1010b5b99db644cb0b08a7d6722384cb33cae3f2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Feb 2021 13:20:17 +0100 Subject: [PATCH 215/310] tools/sched/schedgraph.py: implement __lt__ for EventSource so sorting works. XXX: is coparing y correct? Obtained from: ElectroBSD --- tools/sched/schedgraph.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/sched/schedgraph.py b/tools/sched/schedgraph.py index bfd4339f6633..f7d8b2d4b632 100644 --- a/tools/sched/schedgraph.py +++ b/tools/sched/schedgraph.py @@ -817,6 +817,9 @@ def __eq__(self, other): return False return self.group == other.group and self.name == other.name + def __lt__(self, other): + return self.y < other.y + # It is much faster to append items to a list then to insert them # at the beginning. As a result, we add events in reverse order # and then swap the list during fixup. -- 2.37.1 From 97d23b7a8f36f4976a57c2f596511995a27976a4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 Oct 2021 14:41:05 +0200 Subject: [PATCH 216/310] sys/amd64/conf/ELECTRO_BLOAT: Sync with stable/12 GENERIC Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 54 +++++++++++++++++++++++++----------- 1 file changed, 38 insertions(+), 16 deletions(-) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index ae26c7af8e03..f1dafdfcb67f 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -27,10 +27,15 @@ makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support options SCHED_ULE # ULE scheduler +options NUMA # Non-Uniform Memory Architecture support options PREEMPTION # Enable kernel thread preemption +options VIMAGE # Subsystem virtualization, e.g. VNET options INET # InterNETworking options INET6 # IPv6 communications protocols options TCP_OFFLOAD # TCP offload +options TCP_BLACKBOX # Enhanced TCP event logging +options TCP_HHOOK # hhook(9) framework for TCP +options TCP_RFC7413 # TCP Fast Open options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists @@ -53,8 +58,10 @@ options GEOM_PART_GPT # GUID Partition Tables. # affects other geom classes as well. nooptions GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization +options EFIRT # EFI Runtime Services support options COMPAT_FREEBSD32 # Compatible with i386 binaries options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options COMPAT_FREEBSD11 # Compatible with FreeBSD11 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options STACK # stack(9) support @@ -80,6 +87,13 @@ options RCTL # Resource limits # Debugging support. Always need this: options KDB # Enable kernel debugger support. options KDB_TRACE # Print a stack trace for a panic. + +# Kernel dump features. +options EKCD # Support for encrypted kernel dumps +options GZIO # gzip-compressed kernel and user dumps +options ZSTDIO # zstd-compressed kernel and user dumps +options NETDUMP # netdump(4) client support + # For full debugger support use (turn off in stable branch): options DDB # Support DDB. options GDB # Support remote GDB. @@ -88,6 +102,7 @@ options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones # Make an SMP-capable kernel by default options SMP # Symmetric MultiProcessor Kernel +options EARLY_AP_STARTUP # CPU frequency control device cpufreq @@ -96,22 +111,18 @@ device cpufreq device acpi options ACPI_DMAR device pci +options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers -options ATA_STATIC_ID # Static device numbering device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA # SCSI Controllers device ahc # AHA2940 and onboard AIC7xxx devices -options AHC_REG_PRETTY_PRINT # Print register bitfields in debug - # output. Adds ~128k to driver. device ahd # AHA39320/29320 and onboard AIC79xx devices -options AHD_REG_PRETTY_PRINT # Print register bitfields in debug - # output. Adds ~215k to driver. device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series device mpt # LSI-Logic MPT-Fusion @@ -200,14 +211,18 @@ device ppi # Parallel port interface device device puc # Multi I/O cards and multi-channel UARTs -# PCI Ethernet NICs. -device de # DEC/Intel DC21x4x (``Tulip'') +# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure +device iflib device em # Intel PRO/1000 Gigabit Ethernet Family -device igb # Intel PRO/1000 PCIE Server Gigabit Family +device igc # Intel I225 2.5G Ethernet device ix # Intel PRO/10GbE PCIE PF Ethernet device ixv # Intel PRO/10GbE PCIE VF Ethernet -device ixl # Intel XL710 40Gbe PCIE Ethernet -device ixlv # Intel XL710 40Gbe VF PCIE Ethernet +device ixl # Intel 700 Series Physical Function +device iavf # Intel Adaptive Virtual Function +device vmx # VMware VMXNET3 Ethernet + +# PCI Ethernet NICs. +device de # DEC/Intel DC21x4x (``Tulip'') device le # AMD Am7900 LANCE and Am79C9xx PCnet device ti # Alteon Networks Tigon I/II gigabit Ethernet device txp # 3Com 3cR990 (``Typhoon'') @@ -271,16 +286,18 @@ device ath_rate_sample # SampleRate tx rate control for ath device malo # Marvell Libertas wireless NICs. # Pseudo devices. +device crypto # core crypto support device loop # Network loopback device random # Entropy device device padlock_rng # VIA Padlock RNG device rdrand_rng # Intel Bull Mountain RNG device ether # Ethernet support device vlan # 802.1Q VLAN support -device tun # Packet tunnel. +device tuntap # Packet tunnel. device md # Memory "disks" device gif # IPv6 and IPv4 tunneling device firmware # firmware assist module +device xz # lzma decompression # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! @@ -319,7 +336,10 @@ device virtio_blk # VirtIO Block device device virtio_scsi # VirtIO SCSI device device virtio_balloon # VirtIO Memory Balloon device -# HyperV drivers and enchancement support +# Linux KVM paravirtualization support +device kvm_clock # KVM paravirtual clock driver + +# HyperV drivers and enhancement support device hyperv # HyperV drivers # Xen HVM Guest Optimizations @@ -327,8 +347,10 @@ device hyperv # HyperV drivers options XENHVM # Xen HVM kernel infrastructure device xenpci # Xen HVM Hypervisor services driver -# VMware support -device vmx # VMware VMXNET3 Ethernet +# Netmap provides direct access to TX/RX rings on supported NICs +device netmap # netmap(4) support -# The crypto framework is required by IPSEC -device crypto # Required by IPSEC +# evdev interface +options EVDEV_SUPPORT # evdev support in legacy drivers +device evdev # input event device support +device uinput # install /dev/uinput cdev -- 2.37.1 From 6c4bb1e93434553ee1dda5340f468276dac10080 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 Aug 2022 10:02:48 +0200 Subject: [PATCH 217/310] sys/conf/kern.opts.mk: Add SCTP_SUPPORT to the __DEFAULT_NO_OPTIONS to fix the build Obtained from: ElectroBSD --- sys/conf/kern.opts.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/sys/conf/kern.opts.mk b/sys/conf/kern.opts.mk index 098152ee7c4e..95dd0e0ca570 100644 --- a/sys/conf/kern.opts.mk +++ b/sys/conf/kern.opts.mk @@ -59,6 +59,7 @@ __DEFAULT_NO_OPTIONS = \ NAND \ OFED \ RATELIMIT \ + SCTP_SUPPORT \ SOURCELESS_HOST \ SOURCELESS_UCODE -- 2.37.1 From fece4e7114a3a0d553d8c5b70a6cb460bf2b247f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 27 Oct 2021 19:06:29 +0200 Subject: [PATCH 218/310] sys/modules/Makefile: Don't try to build ice which relies on proprietary firmware Obtained from: ElectroBSD --- sys/modules/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index 94e73b8f9530..7d4684d6bc00 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -606,8 +606,8 @@ _cxgbe= cxgbe .endif .if ${MACHINE_ARCH} == "amd64" || ${MACHINE_ARCH} == "arm64" || ${MACHINE_ARCH:Mpowerpc64*} -_ice= ice .if ${MK_SOURCELESS_UCODE} != "no" +_ice= ice _ice_ddp= ice_ddp .endif .endif -- 2.37.1 From 15a8a7db18472de1e04548b67a07ed05e45c50a4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 27 Oct 2021 19:28:57 +0200 Subject: [PATCH 219/310] sys/modules/Makefile: Only build qatfw if MK_SOURCELESS_UCODE is enabled Obtained from: ElectroBSD --- sys/modules/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index 7d4684d6bc00..dafb9fbbf3ec 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -700,7 +700,9 @@ _ntb= ntb _ocs_fc= ocs_fc _pccard= pccard _qat= qat +.if ${MK_SOURCELESS_UCODE} != "no" _qatfw= qatfw +.endif .if ${MK_OFED} != "no" || defined(ALL_MODULES) _rdma= rdma .endif -- 2.37.1 From 688f7cfdeae6061bb9b12ec209c00325922ce538 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 28 Oct 2021 15:29:16 +0200 Subject: [PATCH 220/310] sys/amd64/conf/ELECTRO_BLOAT: Bring back COMPAT_LINUXKPI based on the assumption that it's needed for drm-kmod Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index f1dafdfcb67f..4eaefc5d07bc 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -114,6 +114,8 @@ device pci options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support +options COMPAT_LINUXKPI + # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers -- 2.37.1 From 9635f4eceb67586b7104981d8f43680227d42e7f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Nov 2021 17:20:36 +0100 Subject: [PATCH 221/310] libexec/rc/rc.d/Makefile: Detach os-release from the build for now (depends on freebsd-version) Obtained from: ElectroBSD --- libexec/rc/rc.d/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index 6819d95a5795..a9959e7e5534 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -72,7 +72,6 @@ CONFS= DAEMON \ ${_nscd} \ ntpdate \ ${_opensm} \ - os-release \ pf \ pflog \ pfsync \ -- 2.37.1 From e226402d21398425ab8b682a532e253d9fdce6e7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Nov 2021 19:14:40 +0100 Subject: [PATCH 222/310] sys/kern: Prevent inlining of chroot_refuse_vdir_fds() Obtained from: ElectroBSD --- sys/kern/kern_descrip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/kern_descrip.c b/sys/kern/kern_descrip.c index 90e01fbdaf1c..085dce1bdc23 100644 --- a/sys/kern/kern_descrip.c +++ b/sys/kern/kern_descrip.c @@ -3864,7 +3864,7 @@ SYSCTL_INT(_kern, OID_AUTO, chroot_allow_open_directories, CTLFLAG_RW, * Helper function for raised chroot(2) security function: Refuse if * any filedescriptors are open directories. */ -static int +static __noinline int chroot_refuse_vdir_fds(struct filedesc *fdp) { struct vnode *vp; -- 2.37.1 From a40b7dec66ded41eca01a7d4ecef1bb8aadfe038 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 3 Nov 2021 09:58:07 +0100 Subject: [PATCH 223/310] Disable REPRODUCIBLE_BUILD for the kernel Obtained from: ElectroBSD --- sys/conf/kern.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/conf/kern.opts.mk b/sys/conf/kern.opts.mk index 95dd0e0ca570..ab8ef6582e5a 100644 --- a/sys/conf/kern.opts.mk +++ b/sys/conf/kern.opts.mk @@ -41,7 +41,6 @@ __DEFAULT_YES_OPTIONS = \ NETGRAPH \ PF \ SPLIT_KERNEL_DEBUG \ - REPRODUCIBLE_BUILD \ TESTS \ USB_GADGET_EXAMPLES \ ZFS @@ -59,6 +58,7 @@ __DEFAULT_NO_OPTIONS = \ NAND \ OFED \ RATELIMIT \ + REPRODUCIBLE_BUILD \ SCTP_SUPPORT \ SOURCELESS_HOST \ SOURCELESS_UCODE -- 2.37.1 From b33866f5d735811e535b04635c1fd7f2d51d3799 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 3 Nov 2021 13:57:40 +0100 Subject: [PATCH 224/310] libc: Add ctime_iso() which returns a date like 2021-11-03 13:36:15 (to be used by zpool) --- contrib/tzcode/stdtime/localtime.c | 11 +++++++++++ include/time.h | 1 + lib/libc/stdtime/Symbol.map | 4 ++++ 3 files changed, 16 insertions(+) diff --git a/contrib/tzcode/stdtime/localtime.c b/contrib/tzcode/stdtime/localtime.c index e221c1fa3964..608842d5d432 100644 --- a/contrib/tzcode/stdtime/localtime.c +++ b/contrib/tzcode/stdtime/localtime.c @@ -1728,6 +1728,17 @@ ctime(const time_t *const timep) return asctime(localtime(timep)); } +static char buf_ctime_iso[21]; + +/* Print a date like 2021-11-03 13:36:15\n */ +char * +ctime_iso(const time_t *const timep) +{ + (void) strftime(buf_ctime_iso, sizeof(buf_ctime_iso), "%Y-%m-%d %H:%M:%S\n", localtime(timep)); + return buf_ctime_iso; +} + + char * ctime_r(const time_t *const timep, char *buf) { diff --git a/include/time.h b/include/time.h index a64a331ee52e..263eb2f81406 100644 --- a/include/time.h +++ b/include/time.h @@ -120,6 +120,7 @@ __BEGIN_DECLS char *asctime(const struct tm *); clock_t clock(void); char *ctime(const time_t *); +char *ctime_iso(const time_t *); #ifndef _STANDALONE double difftime(time_t, time_t); #endif diff --git a/lib/libc/stdtime/Symbol.map b/lib/libc/stdtime/Symbol.map index 3bee3735f7e6..831f87411fc2 100644 --- a/lib/libc/stdtime/Symbol.map +++ b/lib/libc/stdtime/Symbol.map @@ -33,3 +33,7 @@ FBSD_1.0 { asctime_r; asctime; }; + +FBSD_1.1 { + ctime_iso; +}; -- 2.37.1 From 95319f3227f328a438b51d686cdae1248d863f07 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 14:36:15 +0100 Subject: [PATCH 225/310] reproduce.sh: Bump copyright Obtained from: ElectroBSD --- reproduce.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reproduce.sh b/reproduce.sh index 93a525772a9d..ab45f7fd0881 100755 --- a/reproduce.sh +++ b/reproduce.sh @@ -1,7 +1,7 @@ #!/bin/sh ########################################################################## -# Copyright (c) 2015-2016 Fabian Keil +# Copyright (c) 2015-2021 Fabian Keil # # Permission to use, copy, modify, and distribute this software for any # purpose with or without fee is hereby granted, provided that the above -- 2.37.1 From 39cbe1b4e7a94cbc027901bc1d8458314d182cff Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 17:24:55 +0100 Subject: [PATCH 226/310] newsyslog.conf: Do not give world read permissions and don't compress stuff Obtained from: ElectroBSD --- usr.sbin/newsyslog/newsyslog.conf | 39 ++++++++++++++----------------- 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/usr.sbin/newsyslog/newsyslog.conf b/usr.sbin/newsyslog/newsyslog.conf index 80e8270935a1..c8a5ed16dbbd 100644 --- a/usr.sbin/newsyslog/newsyslog.conf +++ b/usr.sbin/newsyslog/newsyslog.conf @@ -8,29 +8,24 @@ # is no process which needs to be signalled when a given log file is # rotated, then the entry for that file should include the 'N' flag. # -# Note: some sites will want to select more restrictive protections than the -# defaults. In particular, it may be desirable to switch many of the 644 -# entries to 640 or 600. For example, some sites will consider the -# contents of maillog, messages, and lpd-errs to be confidential. In the -# future, these defaults may change to more conservative ones. -# # logfilename [owner:group] mode count size when flags [/pid_file] [sig_num] -/var/log/all.log 600 7 * @T00 J -/var/log/auth.log 600 7 1000 @0101T JC -/var/log/console.log 600 5 1000 * J -/var/log/cron 600 3 1000 * JC -/var/log/daily.log 640 7 * @T00 JN -/var/log/debug.log 600 7 1000 * JC -/var/log/init.log 644 3 1000 * J -/var/log/kerberos.log 600 7 1000 * J -/var/log/maillog 640 7 * @T00 JC -/var/log/messages 644 5 1000 @0101T JC -/var/log/monthly.log 640 12 * $M1D0 JN -/var/log/devd.log 644 3 1000 * JC -/var/log/security 600 10 1000 * JC -/var/log/utx.log 644 3 * @01T05 B -/var/log/weekly.log 640 5 * $W6D0 JN -/var/log/daemon.log 644 5 1000 @0101T JC +/var/log/all.log 600 7 * @T00 +/var/log/auth.log 600 7 1000 @0101T C +/var/log/console.log 600 5 1000 * +/var/log/cron 600 3 1000 * C +/var/log/daily.log 640 7 * @T00 N +/var/log/debug.log 600 7 1000 * C +/var/log/init.log 640 3 1000 * +/var/log/kerberos.log 600 7 1000 * +/var/log/maillog 640 7 * @T00 C +/var/log/messages 640 5 1000 @0101T C +/var/log/monthly.log 640 12 * $M1D0 N +/var/log/devd.log 640 3 1000 * C +/var/log/security 600 10 1000 * C +/var/log/utx.log 640 3 * @01T05 B +/var/log/weekly.log 640 5 * $W6D0 N +/var/log/daemon.log 644 0 1000 @0101T C + /etc/newsyslog.conf.d/[!.]*.conf /usr/local/etc/newsyslog.conf.d/[!.]*.conf -- 2.37.1 From 736d6e79235c210a5b32b3c00c87d345c97fc3ce Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 7 Nov 2021 11:45:01 +0100 Subject: [PATCH 227/310] g_eli: Don't include the thread number in the process name if there's only one thread Increases the chances that the provider isn't truncated as shown below (using ElectroBSD-20211105-d40b6832c298): last pid: 83498; load averages: 2.30, 1.84, 1.78 up 0+18:39:06 11:45:39 125 processes: 3 running, 121 sleeping, 1 waiting CPU: 42.2% user, 32.5% nice, 14.9% system, 7.6% interrupt, 2.7% idle Mem: 1125M Active, 406M Inact, 108M Laundry, 2092M Wired, 131M Free ARC: 1020M Total, 365M MFU, 376M MRU, 14M Anon, 17M Header, 248M Other, 1017M Target 468M Compressed, 658M Uncompressed, 1.41:1 Ratio Swap: 16G Total, 294M Used, 16G Free, 1% Inuse PID JID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 4126 4 _tor 4 93 0 1099M 690M CPU1 1 812:03 68.34% /usr/local/bin/tor -f /usr/local/etc/tor/torrc --PidFile /var/run/tor/tor.pid --RunAsDaemon 1 --DataDirectory /var/db/tor 33494 10 _tor 4 37 0 795M 392M kqread 0 296:39 24.45% /usr/local/bin/tor -f /usr/local/etc/tor/torrc --PidFile /var/run/tor/tor.pid --RunAsDaemon 1 --DataDirectory /var/db/tor 12 0 root 18 -52 - 0B 288K WAIT 1 166:22 14.37% [intr] 11 0 root 2 155 ki31 0B 32K RUN 0 549:39 5.24% [idle] 28 0 root 317 -16 - 0B 5088K spa->s 0 2:06 3.89% [zpool-rpool] 25 0 root 1 20 - 0B 16K geli:w 0 1:20 2.30% [g_eli[0] gpt/rpool-] 24 0 root 1 20 - 0B 16K geli:w 1 1:20 2.30% [g_eli[0] gpt/rpool-] 27 0 root 1 20 - 0B 16K geli:w 1 1:20 2.29% [g_eli[0] gpt/rpool-] 26 0 root 1 20 - 0B 16K geli:w 1 1:20 2.29% [g_eli[0] gpt/rpool-] 62912 0 root 1 20 - 0B 16K geli:w 0 18:15 0.92% [g_eli[0] gpt/dpool-] 26878 0 root 1 20 - 0B 16K geli:w 1 18:16 0.91% [g_eli[0] gpt/dpool-] 97173 0 root 1 20 - 0B 16K geli:w 1 18:16 0.90% [g_eli[0] gpt/dpool-] 17460 0 root 1 20 - 0B 16K geli:w 1 18:17 0.88% [g_eli[0] gpt/dpool-] Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 47abbb769a76..c34aee501270 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -970,9 +970,14 @@ g_eli_create(struct gctl_req *req, struct g_class *mp, struct g_provider *bpp, } goto failed; } - - error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, - "g_eli[%u] %s", i, bpp->name); + /* Only include the thread number if there's more than one */ + if (threads == 1) { + error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, + "g_eli %s", bpp->name); + } else { + error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, + "g_eli[%u] %s", i, bpp->name); + } if (error != 0) { g_eli_freesession(wr); free(wr, M_ELI); -- 2.37.1 From cb99d2551358777de0c8b728f36938ebaaebadef Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 28 Oct 2021 14:34:36 +0200 Subject: [PATCH 228/310] release/amd64/make-memstick.sh: Temporarily stop using the freshly build makefs May prevent: sh /usr/src/release/amd64/make-memstick.sh disc1 memstick.img /usr/obj/usr/src/amd64.amd64/release/dist/base/usr/sbin/makefs: Undefined symbol "stat@FBSD_1.5" *** Error code 1 Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 8460ed6f0792..1d52a7338f65 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -53,7 +53,7 @@ MAKEFS_DEBUG_FLAGS=0x81400 # # Bind makefs to a single cpu as the output is core dependant # (XXX: doesn't seem to have the intended effect) -cpuset -l 0 dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ +cpuset -l 0 makefs -T "${EPOCH_DATE-0}" \ -d "${MAKEFS_DEBUG_FLAGS}" \ -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab -- 2.37.1 From 446a94d7197b16afe569738404f950462ce9636a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 28 Oct 2021 14:42:00 +0200 Subject: [PATCH 229/310] release/amd64/make-memstick.sh: Temporarily stop using the freshly build mkimg Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 1d52a7338f65..46fc90474574 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -64,7 +64,7 @@ rm "${2}.mtree" espfilename=$(mktemp /tmp/efiboot.XXXXXX) make_esp_file ${espfilename} ${fat32min} ${1}/boot/loader.efi -dist/base/usr/bin/mkimg -s mbr \ +mkimg -s mbr \ -b ${1}/boot/mbr \ -p efi:=${espfilename} \ -p freebsd:-"mkimg -s bsd -b ${1}/boot/boot -p freebsd-ufs:=${2}.part" \ -- 2.37.1 From 14496355ed47df2a6154388203d2b10e4262583c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 09:16:17 +0100 Subject: [PATCH 230/310] share/man/man5: Don't install os-release.5 which isn't currently available on ElectroBSD Obtained from: ElectroBSD --- share/man/man5/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/share/man/man5/Makefile b/share/man/man5/Makefile index 96fc1e052d95..2ca9af490c21 100644 --- a/share/man/man5/Makefile +++ b/share/man/man5/Makefile @@ -46,7 +46,6 @@ MAN= acct.5 \ nsmb.conf.5 \ nsswitch.conf.5 \ nullfs.5 \ - os-release.5 \ passwd.5 \ pbm.5 \ periodic.conf.5 \ -- 2.37.1 From 86a9972e2828f0a75c959817bd4fe2fc0114ceb0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 09:16:58 +0100 Subject: [PATCH 231/310] etc: Don't create a /etc/os-release symlink as the os-release file generation currently doesn't work Obtained from: ElectroBSD --- etc/Makefile | 2 -- 1 file changed, 2 deletions(-) diff --git a/etc/Makefile b/etc/Makefile index 104e40b6e345..0bb4daddd605 100644 --- a/etc/Makefile +++ b/etc/Makefile @@ -56,8 +56,6 @@ distribution: ${_+_}cd ${.CURDIR}/mtree; ${MAKE} install ${_+_}cd ${SRCTOP}/share/termcap; ${MAKE} etc-termcap ${_+_}cd ${SRCTOP}/usr.sbin/rmt; ${MAKE} etc-rmt - ${INSTALL_SYMLINK} -T "package=runtime" ../var/run/os-release \ - ${DESTDIR}/etc/os-release .if ${MK_UNBOUND} != "no" if [ ! -e ${DESTDIR}/etc/unbound ]; then \ ${INSTALL_SYMLINK} -T "package=unbound" \ -- 2.37.1 From 932114b9de018523c94a9b154205c5d2badd5b8e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 11:21:34 +0100 Subject: [PATCH 232/310] syslogd: Add ElectroBSD output format Obtained from: ElectroBSD --- usr.sbin/syslogd/syslogd.c | 68 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 66 insertions(+), 2 deletions(-) diff --git a/usr.sbin/syslogd/syslogd.c b/usr.sbin/syslogd/syslogd.c index 5c2555480d3f..d75361895347 100644 --- a/usr.sbin/syslogd/syslogd.c +++ b/usr.sbin/syslogd/syslogd.c @@ -407,6 +407,7 @@ static int needdofsync = 0; /* Are any file(s) waiting to be fsynced? */ static struct pidfh *pfh; static int sigpipe[2]; /* Pipe to catch a signal during select(). */ static bool RFC3164OutputFormat = true; /* Use legacy format by default. */ +static bool ElectroBSDOutputFormat = false; static volatile sig_atomic_t MarkSet, WantDie, WantInitialize, WantReapchild; @@ -684,7 +685,10 @@ main(int argc, char *argv[]) resolve = 0; break; case 'O': - if (strcmp(optarg, "bsd") == 0 || + if (strcmp(optarg, "electrobsd") == 0) { + ElectroBSDOutputFormat = true; + RFC3164OutputFormat = false; + } else if (strcmp(optarg, "bsd") == 0 || strcmp(optarg, "rfc3164") == 0) RFC3164OutputFormat = true; else if (strcmp(optarg, "syslog") == 0 || @@ -2033,6 +2037,63 @@ fprintlog_rfc5424(struct filed *f, const char *hostname, const char *app_name, fprintlog_write(f, &il, flags); } + +/* + * Similiar to fprintlog_rfc5424() but the priority is moved + * after the app name so log messages always start with the + * timestamp and are more pleasant to read. + */ +static void +fprintlog_electrobsd(struct filed *f, const char *hostname, const char *app_name, + const char *procid, const char *msgid, const char *structured_data, + const char *msg, int flags) +{ + struct iovlist il; + suseconds_t usec; + int i; + char timebuf[33], priority_number[5]; + + iovlist_init(&il); + if (f->f_type == F_WALL) + iovlist_append(&il, "\r\n\aMessage from syslogd ...\r\n"); + if (strftime(timebuf, sizeof(timebuf), "%FT%T.______%z", + &f->f_lasttime.tm) == sizeof(timebuf) - 2) { + /* Add colon to the time zone offset, which %z doesn't do. */ + timebuf[32] = '\0'; + timebuf[31] = timebuf[30]; + timebuf[30] = timebuf[29]; + timebuf[29] = ':'; + + /* Overwrite space for microseconds with actual value. */ + usec = f->f_lasttime.usec; + for (i = 25; i >= 20; --i) { + timebuf[i] = usec % 10 + '0'; + usec /= 10; + } + iovlist_append(&il, timebuf); + } else + iovlist_append(&il, "-"); + iovlist_append(&il, " "); + iovlist_append(&il, hostname); + iovlist_append(&il, " "); + iovlist_append(&il, app_name == NULL ? "-" : app_name); + iovlist_append(&il, " "); + iovlist_append(&il, "<"); + snprintf(priority_number, sizeof(priority_number), "%d", f->f_prevpri); + iovlist_append(&il, priority_number); + iovlist_append(&il, ">1 "); + iovlist_append(&il, procid == NULL ? "-" : procid); + iovlist_append(&il, " "); + iovlist_append(&il, msgid == NULL ? "-" : msgid); + iovlist_append(&il, " "); + iovlist_append(&il, structured_data == NULL ? "-" : structured_data); + iovlist_append(&il, " "); + iovlist_append(&il, msg); + + fprintlog_write(f, &il, flags); +} + + static void fprintlog_rfc3164(struct filed *f, const char *hostname, const char *app_name, const char *procid, const char *msg, int flags) @@ -2157,7 +2218,10 @@ fprintlog_first(struct filed *f, const char *hostname, const char *app_name, return; } - if (RFC3164OutputFormat) + if (ElectroBSDOutputFormat) + fprintlog_electrobsd(f, hostname, app_name, procid, msgid, + structured_data, msg, flags); + else if (RFC3164OutputFormat) fprintlog_rfc3164(f, hostname, app_name, procid, msg, flags); else fprintlog_rfc5424(f, hostname, app_name, procid, msgid, -- 2.37.1 From 8e03a1023771b368447fd9b05b608a05bcd41dea Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 10:08:48 +0100 Subject: [PATCH 233/310] rc.conf: Let syslogd log in ElectroBSD format by default Obtained from: ElectroBSD --- libexec/rc/rc.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libexec/rc/rc.conf b/libexec/rc/rc.conf index b034ac46e48f..4bbc647b0839 100755 --- a/libexec/rc/rc.conf +++ b/libexec/rc/rc.conf @@ -292,7 +292,7 @@ hostapd_program="/usr/sbin/hostapd" hostapd_enable="NO" # Run hostap daemon. syslogd_enable="YES" # Run syslog daemon (or NO). syslogd_program="/usr/sbin/syslogd" # path to syslogd, if you want a different one. -syslogd_flags="-s" # Flags to syslogd (if enabled). +syslogd_flags="-s -O electrobsd" # Flags to syslogd (if enabled). syslogd_oomprotect="YES" # Don't kill syslogd when swap space is exhausted. altlog_proglist="" # List of chrooted applicatioins in /var inetd_enable="NO" # Run the network daemon dispatcher (YES/NO). -- 2.37.1 From cefd32654ea638e63f49c02916be013223cf444e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 11:52:17 +0100 Subject: [PATCH 234/310] share/mk/bsd.own.mk: Use faster xz compression as we care more about time than space Obtained from: ElectroBSD --- share/mk/bsd.own.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/share/mk/bsd.own.mk b/share/mk/bsd.own.mk index 09d7dfbd25c8..2f63987d0af0 100644 --- a/share/mk/bsd.own.mk +++ b/share/mk/bsd.own.mk @@ -272,9 +272,9 @@ COMPRESS_EXT?= .gz XZ_THREADS?= 0 .if !empty(XZ_THREADS) -XZ_CMD?= xz -T ${XZ_THREADS} +XZ_CMD?= xz -0 -T ${XZ_THREADS} .else -XZ_CMD?= xz +XZ_CMD?= xz -0 .endif PKG_CMD?= pkg -- 2.37.1 From 864e543f0e371766b2aec842bf2a08430a0f5dff Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 7 Nov 2021 15:08:32 +0100 Subject: [PATCH 235/310] sys/geom/eli: If the thread name will be truncated, use the last characters instead of the first ones Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index c34aee501270..4534dfd7703a 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -865,7 +865,9 @@ g_eli_create(struct gctl_req *req, struct g_class *mp, struct g_provider *bpp, struct g_provider *pp; struct g_consumer *cp; struct g_geom_alias *gap; - u_int i, threads; + struct thread dummy_thread; + u_int i, threads, name_offset; + size_t name_len, name_space_available; int dcw, error; G_ELI_DEBUG(1, "Creating device %s%s.", bpp->name, G_ELI_SUFFIX); @@ -970,13 +972,36 @@ g_eli_create(struct gctl_req *req, struct g_class *mp, struct g_provider *bpp, } goto failed; } - /* Only include the thread number if there's more than one */ + name_len = strlen(bpp->name); + name_space_available = sizeof(dummy_thread.td_name) - 1 - + strlen("g_eli"); + if (threads != 1) { + /* + * XXX: Assumes the number of threads + * fits a single digit + */ + name_space_available -= strlen("[0]"); + } + if (name_len < name_space_available) + name_offset = 0; + else + name_offset = name_len - name_space_available + 1; + + G_ELI_DEBUG(1, "geli threads for %s: %d, name_len: %d, " + "name_space_available: %d, name_offset: %d ", + bpp->name, threads, (int)name_len, + (int)name_space_available, name_offset); + + /* + * Only include the thread number if there's more + * than one thread. + */ if (threads == 1) { - error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, - "g_eli %s", bpp->name); + error = kproc_create(g_eli_worker, wr, &wr->w_proc, + 0, 0, "g_eli %s", &bpp->name[name_offset]); } else { - error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, - "g_eli[%u] %s", i, bpp->name); + error = kproc_create(g_eli_worker, wr, &wr->w_proc, + 0, 0, "g_eli[%u] %s", i, &bpp->name[name_offset]); } if (error != 0) { g_eli_freesession(wr); -- 2.37.1 From f08cbbe5e136d3efd94a886516381225d6649bf1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 18:02:40 +0100 Subject: [PATCH 236/310] release/Makefile: Let the memstick ignore geli providers with passphrase Obtained from: ElectroBSD --- release/Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/release/Makefile b/release/Makefile index d35e00bc6f22..34556ec90354 100644 --- a/release/Makefile +++ b/release/Makefile @@ -212,6 +212,9 @@ disc1: packagesystem echo hostname=\"${VOLUME_LABEL}\" >> ${.TARGET}/etc/rc.conf echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf + # Ignore provider with passphrase so booting doesn't require + # console interaction. + echo kern.geom.eli.tries=\"0\" >> ${.TARGET}/boot/loader.conf cp ${.CURDIR}/rc.local ${.TARGET}/etc umask ls -l ${.TARGET}/etc/rc.local -- 2.37.1 From 7144e772fa40a9ebdc8ebf23ed4ec0d55e42fc08 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Nov 2021 16:05:14 +0100 Subject: [PATCH 237/310] Add a ELECTRO_BLOAT_BOOT_PROFILING kernel config Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING diff --git a/sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING b/sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING new file mode 100644 index 000000000000..9447ffa7971d --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING @@ -0,0 +1,5 @@ +include ELECTRO_BLOAT + +ident ELECTRO_BLOAT_BOOT_PROFILING + +options TSLOG -- 2.37.1 From 34268bce66191f0eb6f654602aa3c833bb12c725 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 27 Nov 2015 00:20:59 +0100 Subject: [PATCH 238/310] etc: Register uid and gid for Tor ... so the user doesn't have to do it manually on the host system after installing Tor in a jail. Obtained from: ElectroBSD --- etc/group | 1 + etc/master.passwd | 1 + 2 files changed, 2 insertions(+) diff --git a/etc/group b/etc/group index 2a24f55303ca..f48fef985e4d 100644 --- a/etc/group +++ b/etc/group @@ -33,6 +33,7 @@ audit:*:77: www:*:80: ntpd:*:123: _ypldap:*:160: +_tor:*:256: hast:*:845: tests:*:977: nogroup:*:65533: diff --git a/etc/master.passwd b/etc/master.passwd index b61c13fdd9fe..597aebe125dc 100644 --- a/etc/master.passwd +++ b/etc/master.passwd @@ -24,6 +24,7 @@ auditdistd:*:78:77::0:0:Auditdistd unprivileged user:/var/empty:/usr/sbin/nologi www:*:80:80::0:0:World Wide Web Owner:/nonexistent:/usr/sbin/nologin ntpd:*:123:123::0:0:NTP Daemon:/var/db/ntp:/usr/sbin/nologin _ypldap:*:160:160::0:0:YP LDAP unprivileged user:/var/empty:/usr/sbin/nologin +_tor:*:256:256::0:0:Onion delivery agent:/nonexistent:/usr/sbin/nologin hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin tests:*:977:977::0:0:Unprivileged user for tests:/nonexistent:/usr/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin -- 2.37.1 From 1f8a756384da536e9490e959045926d2b91f3fc3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 11 Nov 2021 03:37:13 +0100 Subject: [PATCH 239/310] rc.conf: Default to syslogd not opening any network sockets Obtained from: ElectroBSD --- libexec/rc/rc.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libexec/rc/rc.conf b/libexec/rc/rc.conf index 4bbc647b0839..b42811a4ddea 100755 --- a/libexec/rc/rc.conf +++ b/libexec/rc/rc.conf @@ -292,7 +292,7 @@ hostapd_program="/usr/sbin/hostapd" hostapd_enable="NO" # Run hostap daemon. syslogd_enable="YES" # Run syslog daemon (or NO). syslogd_program="/usr/sbin/syslogd" # path to syslogd, if you want a different one. -syslogd_flags="-s -O electrobsd" # Flags to syslogd (if enabled). +syslogd_flags="-ss -O electrobsd" # Flags to syslogd (if enabled). syslogd_oomprotect="YES" # Don't kill syslogd when swap space is exhausted. altlog_proglist="" # List of chrooted applicatioins in /var inetd_enable="NO" # Run the network daemon dispatcher (YES/NO). -- 2.37.1 From 4b559b7efab1c542a9dcf5291ca5be967bd3466d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 12 Nov 2021 20:57:48 +0100 Subject: [PATCH 240/310] ELECTRO_BLOAT i386: Sync with GENERIC Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BLOAT | 53 ++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 27 deletions(-) diff --git a/sys/i386/conf/ELECTRO_BLOAT b/sys/i386/conf/ELECTRO_BLOAT index fcfb7c314411..9f9df8483aaa 100644 --- a/sys/i386/conf/ELECTRO_BLOAT +++ b/sys/i386/conf/ELECTRO_BLOAT @@ -12,6 +12,7 @@ makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption +options VIMAGE # Subsystem virtualization, e.g. VNET options INET # InterNETworking options INET6 # IPv6 communications protocols options TCP_OFFLOAD # TCP offload @@ -32,8 +33,8 @@ options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. options GEOM_LABEL # Provides labelization -options COMPAT_FREEBSD9 # Compatible with FreeBSD9 options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options COMPAT_FREEBSD11 # Compatible with FreeBSD11 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options STACK # stack(9) support @@ -64,9 +65,16 @@ options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones +# Kernel dump features. +options EKCD # Support for encrypted kernel dumps +options GZIO # gzip-compressed kernel and user dumps +options ZSTDIO # zstd-compressed kernel and user dumps +options NETDUMP # netdump(4) client support + # To make an SMP kernel, the next two lines are needed options SMP # Symmetric MultiProcessor Kernel device apic # I/O APIC +options EARLY_AP_STARTUP # CPU frequency control device cpufreq @@ -74,22 +82,18 @@ device cpufreq # Bus support. device acpi device pci +options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers -options ATA_STATIC_ID # Static device numbering device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA # SCSI Controllers device ahc # AHA2940 and onboard AIC7xxx devices -options AHC_REG_PRETTY_PRINT # Print register bitfields in debug - # output. Adds ~128k to driver. -device ahd # AHA39320/29320 and onboard AIC79xx devices -options AHD_REG_PRETTY_PRINT # Print register bitfields in debug - # output. Adds ~215k to driver. + device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series device isp # Qlogic family @@ -98,15 +102,6 @@ device mps # LSI-Logic MPT-Fusion 2 device mpr # LSI-Logic MPT-Fusion 3 device trm # Tekram DC395U/UW/F DC315U adapters -device adv # Advansys SCSI adapters -device adw # Advansys wide SCSI adapters -device aha # Adaptec 154x SCSI adapters -device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. -device bt # Buslogic/Mylex MultiMaster SCSI adapters - -device ncv # NCR 53C500 -device nsp # Workbit Ninja SCSI-3 -device stg # TMC 18C30/18C50 device isci # Intel C600 SAS controller # ATA/SCSI peripherals @@ -140,6 +135,10 @@ device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s device pst # Promise Supertrak SX6000 device twe # 3ware ATA RAID +# NVM Express (NVMe) support +device nvme # base NVMe driver +device nvd # expose NVMe namespace as disks, depends on nvme + # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard @@ -162,9 +161,6 @@ device vt_vga device agp # support several AGP chipsets -# Add suspend/resume support for the i8254. -device pmtimer - # PCCARD (PCMCIA) support # PCMCIA and cardbus bridge support device cbb # cardbus (yenta) bridge @@ -183,11 +179,15 @@ device ppi # Parallel port interface device device puc # Multi I/O cards and multi-channel UARTs +# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure +device iflib +device igc # Intel I225 2.5G Ethernet +device em # Intel PRO/1000 Gigabit Ethernet Family +device vmx # VMware VMXNET3 Ethernet + # PCI Ethernet NICs. device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel PRO/1000 Gigabit Ethernet Family -device igb # Intel PRO/1000 PCIE Server Gigabit Family -device ixgb # Intel PRO/10GbE Ethernet Card device le # AMD Am7900 LANCE and Am79C9xx PCnet device ti # Alteon Networks Tigon I/II gigabit Ethernet device txp # 3Com 3cR990 (``Typhoon'') @@ -237,7 +237,6 @@ device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards device ex # Intel EtherExpress Pro/10 and Pro/10+ device ep # Etherlink III based cards device fe # Fujitsu MB8696x based cards -device ie # EtherExpress 8/16, 3C507, StarLAN 10 etc. device sn # SMC's 9000 series of Ethernet chips device xe # Xircom pccard Ethernet @@ -269,7 +268,7 @@ device padlock_rng # VIA Padlock RNG device rdrand_rng # Intel Bull Mountain RNG device ether # Ethernet support device vlan # 802.1Q VLAN support -device tun # Packet tunnel. +device tuntap # Packet tunnel. device md # Memory "disks" device gif # IPv6 and IPv4 tunneling device firmware # firmware assist module @@ -311,7 +310,10 @@ device virtio_blk # VirtIO Block device device virtio_scsi # VirtIO SCSI device device virtio_balloon # VirtIO Memory Balloon device -# HyperV drivers and enchancement support +# Linux KVM paravirtualization support +device kvm_clock # KVM paravirtual clock driver + +# HyperV drivers and enhancement support device hyperv # HyperV drivers # Xen HVM Guest Optimizations @@ -319,9 +321,6 @@ device hyperv # HyperV drivers options XENHVM # Xen HVM kernel infrastructure device xenpci # Xen HVM Hypervisor services driver -# VMware support -device vmx # VMware VMXNET3 Ethernet - # The crypto framework is required by IPSEC device crypto # Required by IPSEC -- 2.37.1 From 2ce0eacffe11b2cc3f1ad03792562a298998018c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 13 Nov 2021 18:02:15 +0100 Subject: [PATCH 241/310] ELECTRO_BLOAT i386: Ditch "options KVA_PAGES=512" It seems to break the build and it looks like it's no longer needed anyway. [fk@elektrobier3 ~]$ time BUILD_JAIL=ElectroBSD-i386 sh /usr/src/reproduce-electrobsd.sh [...] cc -target i386-unknown-freebsd12.3 --sysroot=/usr/obj/usr/src/i386.i386/tmp -B/usr/obj/usr/src/i386.i386/tmp/usr/bin -c -x assembler-with-cpp -DLOCORE -O -pipe -g -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/ck/include -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -MD -MF.depend.locore.o -MTlocore.o -fdebug-prefix-map=./machine=/usr/src/sys/i386/include -fdebug-prefix-map=./x86=/usr/src/sys/x86/include -mno-mmx -mno-sse -msoft-float -ffreestanding -fwrapv -fstack-protector -gdwarf-2 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error=tautological-compare -Wno-error=empty-body -Wno-error=parentheses-equality -Wno-error=unused-function -Wno-error=pointer-sign -Wno-error=shift-negative-value -Wno-address-of-packed-member -Wno-format-zero-length -mno-aes -mno-avx -std=iso9899:1999 -Werror /usr/src/sys/i386/i386/locore.s In file included from /usr/src/sys/i386/i386/locore.s:51: ./machine/pmap.h:118:9: error: 'KVA_PAGES' macro redefined [-Werror,-Wmacro-redefined] #define KVA_PAGES (256*4) ^ ./opt_global.h:9:9: note: previous definition is here #define KVA_PAGES 512 ^ 1 error generated. *** Error code 1 Stop. make[2]: stopped in /usr/obj/usr/src/i386.i386/sys/ELECTRO_BLOAT *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src 31.22s real 11.87s user 6.65s sys 43148 maximum resident set size 1865 average shared memory size 54 average unshared data size 128 average unshared stack size 417404 page reclaims 1163 page faults 0 swaps 53338 block input operations 126 block output operations 0 messages sent 0 messages received 0 signals received 12584 voluntary context switches 46442 involuntary context switches ElectroBSD-i386: removed Script done, output file is /var/log/buildlogs/reprolog-2021-11-12_20:58 Script started, output file is /var/log/buildlogs/reprolog-2021-11-12_20:58 Script done, output file is /var/log/buildlogs/reprolog-2021-11-12_20:58 real 641m19.404s user 712m47.369s sys 24m25.377s Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BLOAT | 4 ---- 1 file changed, 4 deletions(-) diff --git a/sys/i386/conf/ELECTRO_BLOAT b/sys/i386/conf/ELECTRO_BLOAT index 9f9df8483aaa..efeb7393f394 100644 --- a/sys/i386/conf/ELECTRO_BLOAT +++ b/sys/i386/conf/ELECTRO_BLOAT @@ -328,7 +328,3 @@ device crypto # Required by IPSEC # For details see UPDATING entry 20121223. After r286288 it's # probably no longer necessary, but for now we keep it anyway. options KSTACK_PAGES=4 - -# Increase the size of the kernel virtual address space -# so ZFS can cache more stuff. -options KVA_PAGES=512 -- 2.37.1 From ffcf52b27b27f9caf1059783fcc2e3bdd51ff00f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 13:27:28 +0100 Subject: [PATCH 242/310] ggated: Ignore SIGPIPE to prevent DoS ... by a single prematurely closed client connection. Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 7cacbf58037e..bdb24d19b94c 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1053,6 +1053,7 @@ main(int argc, char *argv[]) pidfile_write(pfh); signal(SIGCHLD, SIG_IGN); + signal(SIGPIPE, SIG_IGN); sfd = socket(AF_INET, SOCK_STREAM, 0); if (sfd == -1) -- 2.37.1 From 09435bc4e8e585976ed4725b27529c1d89925538 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 13:47:31 +0100 Subject: [PATCH 243/310] ggated: Remove connection if the initial packet couldn't be sent Should help to mitigate DoS after flooding ggated with incomplete requests: error: accept(): Software caused connection abort. error: Exiting. Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index bdb24d19b94c..95d167d062f9 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -956,6 +956,7 @@ handshake(struct sockaddr *from, int sfd) if (data == -1) { sendfail(sfd, errno, "Error while sending initial packet: %s.", strerror(errno)); + connection_remove(conn); return (0); } -- 2.37.1 From faaf2b02ccb9530475519f943696e080076a4f65 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 14:09:24 +0100 Subject: [PATCH 244/310] ggated: Continue if accept() is interrupted or the remote connection is lost Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 95d167d062f9..72720bfe2ba7 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1078,9 +1078,11 @@ main(int argc, char *argv[]) for (;;) { fromlen = sizeof(from); tmpsfd = accept(sfd, &from, &fromlen); - if (tmpsfd == -1) + if (tmpsfd == -1) { + if (errno == EINTR || errno == ECONNABORTED) + continue; g_gate_xlog("accept(): %s.", strerror(errno)); - + } if (got_sighup) { got_sighup = 0; exports_get(); -- 2.37.1 From 543307f64df9b594d4c13712b66d64ac808c784c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 14:16:47 +0100 Subject: [PATCH 245/310] ggated: Prevent c_diskfd leaks through connection_remove() Should help against DoS: [...] debug: Connection created [127.0.0.1, /tank/scratch/testfile]. debug: New connection created (token=2197914058). debug: exports[/tank/scratch/testfile2]: Path mismatch. debug: Sending initial packet. error: accept(): Too many open files. error: Exiting. Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 72720bfe2ba7..1fddbabd650d 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -527,6 +527,8 @@ connection_remove(struct ggd_connection *conn) close(conn->c_sendfd); if (conn->c_recvfd != -1) close(conn->c_recvfd); + if (conn->c_diskfd != -1) + close(conn->c_diskfd); free(conn->c_path); free(conn); } -- 2.37.1 From b567ae9b366b2b2d3dafe767c3792a1497140ffc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 15:52:39 +0100 Subject: [PATCH 246/310] ggated: Check for connection_add() failures properly Prevents a socket leak Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 1fddbabd650d..f49949d510c2 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -918,7 +918,7 @@ handshake(struct sockaddr *from, int sfd) */ g_gate_log(LOG_DEBUG, "Found existing connection (token=%lu).", (unsigned long)conn->c_token); - if (connection_add(conn, &cinit, from, sfd) == -1) { + if (connection_add(conn, &cinit, from, sfd) == EEXIST) { connection_remove(conn); return (0); } -- 2.37.1 From fa7fdccd8a2c1e5c1ba6fe68854bd9fb1f462faa Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 2 Apr 2015 15:24:58 +0200 Subject: [PATCH 247/310] ggated recv_thread(): Do not queue incomplete WRITE requests Verifying that g_gate_recv()'s return code isn't -1 is insufficient as it's a thin wrapper arround recv(2) which, quoting its man page, "may still return less data than requested if a signal is caught, an error or disconnect occurs, or the next data to be received is of a different type than that returned". Previously incomplete WRITE requests would be scheduled with partially uninitialized memory, potentially resulting in file system corruption or, worse, bogus data being later on returned as valid. Security impact: A MITM may cause data corruption by disrupting the connection from ggatec's send_thread() to ggated's recv_thread() at the right point in time. This does not require access to the plain text traffic but if encryption is involved the attacker would have to guess that it's ggate traffic and disrupt connections blindly, hoping that some of the disruptions trigger the bug. The issue was discovered after ZFS on the ggatec side reported checksum errors which weren't reproducible on the ggated side where ZFS had received and checksummed bogus data. The ggate traffic was tunneled through SSH and Tor with sshd running as Tor location hidden service. Reported to security-officer@FreeBSD.org on 2015-04-05. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index f49949d510c2..203bf4474a14 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -684,6 +684,9 @@ recv_thread(void *arg) if (data == -1) { g_gate_xlog("Error while receiving data: %s.", strerror(errno)); + } else if ((uint32_t)data != req->r_length) { + g_gate_xlog("Received %d bytes of data while " + "expecting %u.", data, req->r_length); } } -- 2.37.1 From 55a1d4f3596d1cbf2ce1517a808e9d690f43ae9b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 2 Apr 2015 12:09:40 +0200 Subject: [PATCH 248/310] ggated recv_thread(): Do not queue requests with invalid values ... that would cause abort()s when read by the disk_thread() later on. From ggatec's point of view it doesn't make a difference as the connection will get closed either way, but at least the admin on the server side doesn't have to deal with core dumps. Security impact: An authenticated attacker may intentionally cause the ggated process that handles the attacker's connection to core dump and thus use more disk space than intentionally provisioned by the server admin. Without the following patch ggated core dumps may require more than 100 GB of disk space. Reported to security-officer@FreeBSD.org on 2015-04-05. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 203bf4474a14..935b289868ea 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -668,6 +668,23 @@ recv_thread(void *arg) g_gate_log(LOG_DEBUG, "%s: offset=%" PRIu64 " length=%" PRIu32, __func__, req->r_offset, req->r_length); + /* + * Reject requests that violate assertions in disk_thread(). + */ + if (req->r_cmd != GGATE_CMD_READ && + req->r_cmd != GGATE_CMD_WRITE) { + g_gate_xlog("Request contains invalid command."); + } + if (req->r_offset + req->r_length > + (uintmax_t)conn->c_mediasize) { + g_gate_xlog("Request out of bounds."); + } + if (req->r_offset % conn->c_sectorsize != 0 || + req->r_length % conn->c_sectorsize != 0) { + g_gate_xlog("Request length or offset does " + "not fit sector size."); + } + /* * Allocate memory for data. */ -- 2.37.1 From dbcf3f1d755fbd5d7b5b15725f07f3776729d0a6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 2 Apr 2015 19:52:54 +0200 Subject: [PATCH 249/310] ggated recv_thread(): Reject request with more than MAXPHYS bytes of data .. to limit the amount of memory we (try to) allocate on behalf of the client without knowing whether or not the client actually intents to use it. MAXPHYS is the hardcoded limit in ggatec so anything above it is suspicious and could be a DoS attempt. This commit forces users who like to tune MAXPHYS to make sure the value used by ggated is not below the one used by ggatec. While not ideal, this seems preferable to the DoS risk. Reported to security-officer@FreeBSD.org on 2015-04-05. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 935b289868ea..d5d8d564253c 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -685,6 +685,16 @@ recv_thread(void *arg) "not fit sector size."); } + /* + * Limit the amount of memory we allocate on behalf of + * the client. MAXPHYS is the hard limit in ggatec, + * values above it are thus pretty suspicious. + */ + if (req->r_length > MAXPHYS) { + g_gate_xlog("Request length above MAXPHYS: %u > %u", + (unsigned)req->r_length, MAXPHYS); + } + /* * Allocate memory for data. */ -- 2.37.1 From 1b279655097d94c9681ccfe243128c24d9226fec Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 24 Apr 2015 14:04:31 +0200 Subject: [PATCH 250/310] ggatec: Add support for SOCKS5 with domain names Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 104 +++++++++++++++++++++++++++++++++++-- 1 file changed, 99 insertions(+), 5 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 0de8504ce3c4..df00a144dca8 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -67,6 +67,8 @@ static unsigned flags = 0; static int force = 0; static unsigned queue_size = G_GATE_QUEUE_SIZE; static unsigned port = G_GATE_PORT; +static char *socks_dest = NULL; +static unsigned dest_port = 3080; static off_t mediasize; static unsigned sectorsize = 0; static unsigned timeout = G_GATE_TIMEOUT; @@ -82,9 +84,11 @@ usage(void) fprintf(stderr, "usage: %s create [-nv] [-o ] [-p port] " "[-q queue_size] [-R rcvbuf] [-S sndbuf] [-s sectorsize] " - "[-t timeout] [-u unit] \n", getprogname()); + "[-t timeout] [-T :] [-u unit] \n", + getprogname()); fprintf(stderr, " %s rescue [-nv] [-o ] [-p port] " - "[-R rcvbuf] [-S sndbuf] <-u unit> \n", getprogname()); + "[-R rcvbuf] [-S sndbuf] [-T :] <-u unit> " + " \n", getprogname()); fprintf(stderr, " %s destroy [-f] <-u unit>\n", getprogname()); fprintf(stderr, " %s list [-v] [-u unit]\n", getprogname()); exit(EXIT_FAILURE); @@ -286,6 +290,69 @@ recv_thread(void *arg __unused) pthread_exit(NULL); } +static void +negotiate_socks_connection(int sfd) +{ + struct negotiation_request { + char version; + char nmethods; + char method; + } neg_request; + struct socks_request { + char version; + char cmd; + char reserved; + char address_type; + char host_length; + char dest[255 + 2]; + } socks_request; + char response[10]; + size_t request_length; + size_t host_length; + + host_length = strlen(socks_dest); + + neg_request.version = '\x05'; + neg_request.nmethods = '\x01'; /* We support one method: */ + neg_request.method = '\x00'; /* no authentication */ + + g_gate_log(LOG_DEBUG, "Starting SOCKS negotiation."); + if (g_gate_send(sfd, &neg_request, sizeof(neg_request), MSG_NOSIGNAL) == -1) + g_gate_xlog("Failed to send SOCKS negotiation request."); + + if (g_gate_recv(sfd, &response, sizeof(response), MSG_WAITALL) != 2) + g_gate_xlog("Failed to read SOCKS negotiation response."); + + if (response[0] != '\x05' || response[1] != '\x00') + g_gate_xlog("SOCKS negotiation failed."); + + g_gate_log(LOG_DEBUG, "Negotiated SOCKS5. " + "Requesting connection to %s:%d.", socks_dest, dest_port); + + socks_request.version = '\x05'; + socks_request.cmd = '\x01'; /* Connect */ + socks_request.reserved = '\x00'; + socks_request.address_type = '\x03'; /* Address is domain name */; + socks_request.host_length = (char)host_length; + strncpy(socks_request.dest, socks_dest, host_length); + socks_request.dest[host_length] = (char)((dest_port >> 8) & 0xff); + socks_request.dest[host_length + 1] = (char)(dest_port & 0xff); + request_length = sizeof(socks_request) - sizeof(socks_request.dest) + + host_length + 2; + + if (g_gate_send(sfd, &socks_request, request_length, MSG_NOSIGNAL) == -1) + g_gate_xlog("Failed to send SOCKS5 request."); + + if (g_gate_recv(sfd, &response, sizeof(response), MSG_WAITALL) != sizeof(response)) + g_gate_xlog("Failed to read SOCKS5 response."); + + if (response[0] != '\x05' || response[1] != '\x00') + g_gate_xlog("Failed to SOCKS5 connect to %s:%d", + socks_dest, dest_port); + + g_gate_log(LOG_INFO, "Connected to: %s:%d.", socks_dest, dest_port); +} + static int handshake(int dir) { @@ -324,6 +391,9 @@ handshake(int dir) g_gate_log(LOG_INFO, "Connected to the server: %s:%d.", host, port); + if (socks_dest != NULL) + negotiate_socks_connection(sfd); + /* * Create and send version packet. */ @@ -503,8 +573,13 @@ g_gatec_create(void) ggioc.gctl_maxcount = queue_size; ggioc.gctl_timeout = timeout; ggioc.gctl_unit = unit; - snprintf(ggioc.gctl_info, sizeof(ggioc.gctl_info), "%s:%u %s", host, - port, path); + if (socks_dest != NULL) + snprintf(ggioc.gctl_info, sizeof(ggioc.gctl_info), + "socks5://%s:%u -> %s:%u %s", host, + port, socks_dest, dest_port, path); + else + snprintf(ggioc.gctl_info, sizeof(ggioc.gctl_info), "%s:%u %s", + host, port, path); g_gate_ioctl(G_GATE_CMD_CREATE, &ggioc); if (unit == -1) { printf("%s%u\n", G_GATE_PROVIDER_NAME, ggioc.gctl_unit); @@ -563,8 +638,9 @@ main(int argc, char *argv[]) argv += 1; for (;;) { int ch; + char *p; - ch = getopt(argc, argv, "fno:p:q:R:S:s:t:u:v"); + ch = getopt(argc, argv, "fno:p:q:R:S:s:t:T:u:v"); if (ch == -1) break; switch (ch) { @@ -632,6 +708,24 @@ main(int argc, char *argv[]) if (sectorsize == 0 && errno != 0) errx(EXIT_FAILURE, "Invalid sectorsize."); break; + case 'T': + if (action != CREATE && action != RESCUE) + usage(); + socks_dest = optarg; + p = strchr(socks_dest, ':'); + if (p != NULL) { + errno = 0; + *p = '\0'; + p++; + dest_port = strtoul(p, NULL, 10); + if (dest_port == 0 && errno != 0) + errx(EXIT_FAILURE, + "Invalid socks5t port: %s.", p); + } + if (strlen(socks_dest) > (size_t)255) + errx(EXIT_FAILURE, + "Socks destination address too long."); + break; case 't': if (action != CREATE) usage(); -- 2.37.1 From e8700482c98c01f0426421739edfef99c326bd4c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 24 Apr 2015 15:26:42 +0200 Subject: [PATCH 251/310] ggatec.8: Document SOCKS5 support Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.8 | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/sbin/ggate/ggatec/ggatec.8 b/sbin/ggate/ggatec/ggatec.8 index 6f761dcfd99b..ba3c9ecb6f0b 100644 --- a/sbin/ggate/ggatec/ggatec.8 +++ b/sbin/ggate/ggatec/ggatec.8 @@ -41,6 +41,7 @@ .Op Fl R Ar rcvbuf .Op Fl S Ar sndbuf .Op Fl s Ar sectorsize +.Op Fl T Ar remote_target:port .Op Fl t Ar timeout .Op Fl u Ar unit .Ar host @@ -53,6 +54,7 @@ .Op Fl p Ar port .Op Fl R Ar rcvbuf .Op Fl S Ar sndbuf +.Op Fl T Ar remote_target:port .Fl u Ar unit .Ar host .Ar path @@ -137,6 +139,9 @@ Sector size for .Nm ggate provider. If not specified, it is taken from the device, or set to 512 bytes for files. +.It Fl T Ar remote_host:port +Use SOCK5 to open connection to remote_host:port before switching +to the ggated protocol. .It Fl t Ar timeout Number of seconds to wait before an I/O request will be canceled. Default is 0, which means no timeout. @@ -167,6 +172,14 @@ server# ggated client# ggatec create -o ro server /dev/cd0 ggate0 client# mount_cd9660 /dev/ggate0 /cdrom + +.Ed +Connect to 127.0.1.1:9050, SOCKS5-negotiate a connection to +the Tor location hidden service czdqtfrgvizltdal.onion:1312 +and access a ZVOL: +.Bd -literal -offset indent +# ggatec create -T czdqtfrgvizltdal.onion:1312 -p 9050 \\ + 127.0.1.1 /dev/zvol/dpool/ggated/czdqtfrgvizltdal.eli .Ed .Sh SEE ALSO .Xr geom 4 , -- 2.37.1 From 32d309b7545031ecbba57f6e4f5dda87357ae173 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 27 Apr 2015 19:10:17 +0200 Subject: [PATCH 252/310] ggatec: Reject unexpected GGATE commands in recv_thread() Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index df00a144dca8..989eed6ed2c5 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -266,6 +266,10 @@ recv_thread(void *arg __unused) break; } } + if (ggio.gctl_cmd != GGATE_CMD_READ && + ggio.gctl_cmd != GGATE_CMD_WRITE) { + g_gate_xlog("Unexpected GGATE_CMD: %d", ggio.gctl_cmd); + } if (ggio.gctl_error == 0 && ggio.gctl_cmd == GGATE_CMD_READ) { numbytesprocd = g_gate_recv(recvfd, ggio.gctl_data, -- 2.37.1 From d941b4a927b3ac6440901603b9d06da2f1c948af Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 27 Apr 2015 19:15:18 +0200 Subject: [PATCH 253/310] ggatec: Log if the remote side signals errors Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 989eed6ed2c5..78daab7d50b1 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -271,6 +271,12 @@ recv_thread(void *arg __unused) g_gate_xlog("Unexpected GGATE_CMD: %d", ggio.gctl_cmd); } + if (ggio.gctl_error != 0) { + g_gate_log(LOG_ERR, + "Remote side signaled error %d: %s.", + ggio.gctl_error, strerror(ggio.gctl_error)); + } + if (ggio.gctl_error == 0 && ggio.gctl_cmd == GGATE_CMD_READ) { numbytesprocd = g_gate_recv(recvfd, ggio.gctl_data, ggio.gctl_length, MSG_WAITALL); -- 2.37.1 From 825ce5330d5ee68f90fe9a442c44db3e20e2d859 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 27 Apr 2015 19:53:30 +0200 Subject: [PATCH 254/310] ggate[cd]: Improve BIO_FLUSH support (XXX) Let ggated transform BIO_FLUSH requests into fsync() calls. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 5 ++++- sbin/ggate/ggated/ggated.c | 21 +++++++++++++++------ 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 78daab7d50b1..19b90081d772 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -163,7 +163,9 @@ send_thread(void *arg __unused) hdr.gh_cmd = GGATE_CMD_WRITE; break; case BIO_FLUSH: + g_gate_log(LOG_DEBUG, "FLUSH request"); hdr.gh_cmd = GGATE_CMD_FLUSH; + assert(ggio.gctl_length == 0); break; default: g_gate_log(LOG_NOTICE, "Unknown gctl_cmd: %i", @@ -267,7 +269,8 @@ recv_thread(void *arg __unused) } } if (ggio.gctl_cmd != GGATE_CMD_READ && - ggio.gctl_cmd != GGATE_CMD_WRITE) { + ggio.gctl_cmd != GGATE_CMD_WRITE && + ggio.gctl_cmd != GGATE_CMD_FLUSH) { g_gate_xlog("Unexpected GGATE_CMD: %d", ggio.gctl_cmd); } diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index d5d8d564253c..820984add062 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -672,8 +672,10 @@ recv_thread(void *arg) * Reject requests that violate assertions in disk_thread(). */ if (req->r_cmd != GGATE_CMD_READ && - req->r_cmd != GGATE_CMD_WRITE) { - g_gate_xlog("Request contains invalid command."); + req->r_cmd != GGATE_CMD_WRITE && + req->r_cmd != GGATE_CMD_FLUSH) { + g_gate_xlog("Request contains invalid command: %d", + req->r_cmd); } if (req->r_offset + req->r_length > (uintmax_t)conn->c_mediasize) { @@ -696,9 +698,10 @@ recv_thread(void *arg) } /* - * Allocate memory for data. + * Allocate memory for data, except when flushing. */ - req->r_data = malloc_waitok(req->r_length); + req->r_data = req->r_cmd != GGATE_CMD_FLUSH ? + malloc_waitok(req->r_length) : NULL; /* * Receive data to write for WRITE request. @@ -758,6 +761,9 @@ disk_thread(void *arg) /* * Check the request. */ + assert(req->r_cmd == GGATE_CMD_READ || + req->r_cmd == GGATE_CMD_WRITE || + req->r_cmd == GGATE_CMD_FLUSH); assert(req->r_offset + req->r_length <= (uintmax_t)conn->c_mediasize); assert((req->r_offset % conn->c_sectorsize) == 0); assert((req->r_length % conn->c_sectorsize) == 0); @@ -782,9 +788,12 @@ disk_thread(void *arg) req->r_data = NULL; break; case GGATE_CMD_FLUSH: - data = fsync(fd); - if (data != 0) + g_gate_log(LOG_DEBUG, "Flushing"); + if (fsync(fd)) { req->r_error = errno; + g_gate_log(LOG_ERR, "Flushing failed: %s", + strerror(errno)); + } break; default: g_gate_log(LOG_DEBUG, "Unsupported request: %i", req->r_cmd); -- 2.37.1 From 2b9120860ebbd9cb2e3595ec5e9a5e940784a57f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 29 Apr 2015 12:44:56 +0200 Subject: [PATCH 255/310] ggatec: Log the command type for hdr packets (when debugging) ... and provide more details about failed requests. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 10 +++++++--- sbin/ggate/shared/ggate.c | 16 ++++++++++++++++ sbin/ggate/shared/ggate.h | 1 + 3 files changed, 24 insertions(+), 3 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 19b90081d772..2e08189ac220 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -182,7 +182,8 @@ send_thread(void *arg __unused) g_gate_swap2n_hdr(&hdr); numbytesprocd = g_gate_send(sendfd, &hdr, sizeof(hdr), MSG_NOSIGNAL); - g_gate_log(LOG_DEBUG, "Sent hdr packet."); + g_gate_log(LOG_DEBUG, "Sent hdr packet (%s).", + g_gate_cmd2str(hdr.gh_cmd)); g_gate_swap2h_hdr(&hdr); if (reconnect) break; @@ -248,7 +249,8 @@ recv_thread(void *arg __unused) pthread_kill(sendtd, SIGUSR1); break; } - g_gate_log(LOG_DEBUG, "Received hdr packet."); + g_gate_log(LOG_DEBUG, "Received hdr packet (%s).", + g_gate_cmd2str(hdr.gh_cmd)); ggio.gctl_seq = hdr.gh_seq; ggio.gctl_cmd = hdr.gh_cmd; @@ -276,7 +278,9 @@ recv_thread(void *arg __unused) if (ggio.gctl_error != 0) { g_gate_log(LOG_ERR, - "Remote side signaled error %d: %s.", + "%s for %d bytes at offset %d failed. " + "Error %d: %s.", g_gate_cmd2str(ggio.gctl_cmd), + ggio.gctl_length, ggio.gctl_offset, ggio.gctl_error, strerror(ggio.gctl_error)); } diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index 2f544f7a2c9c..6197540d9678 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -409,3 +409,19 @@ g_gate_str2ip(const char *str) return (INADDR_NONE); return (((struct in_addr *)(void *)hp->h_addr)->s_addr); } + +const char * +g_gate_cmd2str(int cmd) +{ + + switch (cmd) { + case GGATE_CMD_READ: + return ("GGATE_CMD_READ"); + case GGATE_CMD_WRITE: + return ("GGATE_CMD_WRITE"); + case GGATE_CMD_FLUSH: + return ("GGATE_CMD_FLUSH"); + } + + return ("unknown (invalid?) GGATE command"); +} diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index d399b247cd75..846abb7f30bd 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -114,6 +114,7 @@ void g_gate_socket_settings(int sfd); void g_gate_list(int unit, int verbose); #endif in_addr_t g_gate_str2ip(const char *str); +const char *g_gate_cmd2str(int cmd); /* * g_gate_swap2h_* - functions swap bytes to host byte order (from big endian). -- 2.37.1 From c66ab52335c60938014b25611b44038cecdcb14f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 6 May 2015 15:55:08 +0200 Subject: [PATCH 256/310] ggated disk_thread(): Include the command in the debug output Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 820984add062..85c91f412d72 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -768,8 +768,10 @@ disk_thread(void *arg) assert((req->r_offset % conn->c_sectorsize) == 0); assert((req->r_length % conn->c_sectorsize) == 0); - g_gate_log(LOG_DEBUG, "%s: offset=%" PRIu64 " length=%" PRIu32, - __func__, req->r_offset, req->r_length); + g_gate_log(LOG_DEBUG, + "%s: cmd=%s offset=%" PRIu64 " length=%" PRIu32, + __func__, g_gate_cmd2str(req->r_cmd), req->r_offset, + req->r_length); /* * Do the request. -- 2.37.1 From 58f0e1dbb7f8b01ab272c9b808489fafb5827ebd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 5 May 2015 17:39:16 +0200 Subject: [PATCH 257/310] ggate[cd]: Add BIO_DELETE support On the ggated side the requests are translated into writes of zero which ZFS will convert into BIO_DELETE requests again when zle compression is enabled. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 5 ++++ sbin/ggate/ggated/ggated.c | 47 ++++++++++++++++++++++++++++++++------ sbin/ggate/shared/ggate.c | 2 ++ sbin/ggate/shared/ggate.h | 1 + 4 files changed, 48 insertions(+), 7 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 2e08189ac220..f6ba77a9d034 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -162,6 +162,10 @@ send_thread(void *arg __unused) case BIO_WRITE: hdr.gh_cmd = GGATE_CMD_WRITE; break; + case BIO_DELETE: + g_gate_log(LOG_DEBUG, "DELETE request"); + hdr.gh_cmd = GGATE_CMD_DELETE; + break; case BIO_FLUSH: g_gate_log(LOG_DEBUG, "FLUSH request"); hdr.gh_cmd = GGATE_CMD_FLUSH; @@ -272,6 +276,7 @@ recv_thread(void *arg __unused) } if (ggio.gctl_cmd != GGATE_CMD_READ && ggio.gctl_cmd != GGATE_CMD_WRITE && + ggio.gctl_cmd != GGATE_CMD_DELETE && ggio.gctl_cmd != GGATE_CMD_FLUSH) { g_gate_xlog("Unexpected GGATE_CMD: %d", ggio.gctl_cmd); } diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 85c91f412d72..cd36dc853c2a 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -651,6 +651,7 @@ recv_thread(void *arg) * Get header packet. */ req = malloc_waitok(sizeof(*req)); + memset(req, 0, sizeof(*req)); data = g_gate_recv(fd, &req->r_hdr, sizeof(req->r_hdr), MSG_WAITALL); if (data == 0) { @@ -673,6 +674,7 @@ recv_thread(void *arg) */ if (req->r_cmd != GGATE_CMD_READ && req->r_cmd != GGATE_CMD_WRITE && + req->r_cmd != GGATE_CMD_DELETE && req->r_cmd != GGATE_CMD_FLUSH) { g_gate_xlog("Request contains invalid command: %d", req->r_cmd); @@ -692,21 +694,16 @@ recv_thread(void *arg) * the client. MAXPHYS is the hard limit in ggatec, * values above it are thus pretty suspicious. */ - if (req->r_length > MAXPHYS) { + if (req->r_length > MAXPHYS && req->r_cmd != GGATE_CMD_DELETE) { g_gate_xlog("Request length above MAXPHYS: %u > %u", (unsigned)req->r_length, MAXPHYS); } - /* - * Allocate memory for data, except when flushing. - */ - req->r_data = req->r_cmd != GGATE_CMD_FLUSH ? - malloc_waitok(req->r_length) : NULL; - /* * Receive data to write for WRITE request. */ if (req->r_cmd == GGATE_CMD_WRITE) { + req->r_data = malloc_waitok(req->r_length); g_gate_log(LOG_DEBUG, "Waiting for %u bytes of data...", req->r_length); data = g_gate_recv(fd, req->r_data, req->r_length, @@ -733,6 +730,34 @@ recv_thread(void *arg) } } +static ssize_t +delete_range(int fd, size_t length, off_t offset) +{ + static char zeros[MAXPHYS]; + size_t written; + + written = 0; + + do + { + int ret; + size_t bytes_left; + size_t chunk_size; + + bytes_left = length - written; + chunk_size = bytes_left > MAXPHYS ? MAXPHYS : bytes_left; + ret = pwrite(fd, zeros, chunk_size, offset + written); + if (ret == -1) + return (written); + written += ret; + } while (written < length); + + g_gate_log(LOG_DEBUG, "Overwritten %u bytes at offset %jd with zeros", + written, (intmax_t)offset); + + return (written); +} + static void * disk_thread(void *arg) { @@ -763,6 +788,7 @@ disk_thread(void *arg) */ assert(req->r_cmd == GGATE_CMD_READ || req->r_cmd == GGATE_CMD_WRITE || + req->r_cmd == GGATE_CMD_DELETE || req->r_cmd == GGATE_CMD_FLUSH); assert(req->r_offset + req->r_length <= (uintmax_t)conn->c_mediasize); assert((req->r_offset % conn->c_sectorsize) == 0); @@ -779,9 +805,16 @@ disk_thread(void *arg) data = 0; switch (req->r_cmd) { case GGATE_CMD_READ: + assert(req->r_data == NULL); + req->r_data = malloc_waitok(req->r_length); data = pread(fd, req->r_data, req->r_length, req->r_offset); break; + case GGATE_CMD_DELETE: + data = delete_range(fd, req->r_length, + req->r_offset); + assert((size_t)data <= req->r_length); + break; case GGATE_CMD_WRITE: data = pwrite(fd, req->r_data, req->r_length, req->r_offset); diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index 6197540d9678..d6be4948d9c6 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -419,6 +419,8 @@ g_gate_cmd2str(int cmd) return ("GGATE_CMD_READ"); case GGATE_CMD_WRITE: return ("GGATE_CMD_WRITE"); + case GGATE_CMD_DELETE: + return ("GGATE_CMD_DELETE"); case GGATE_CMD_FLUSH: return ("GGATE_CMD_FLUSH"); } diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 846abb7f30bd..f2b602670088 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -58,6 +58,7 @@ #define GGATE_CMD_READ 0 #define GGATE_CMD_WRITE 1 #define GGATE_CMD_FLUSH 3 +#define GGATE_CMD_DELETE 4 extern int g_gate_devfd; extern int g_gate_verbose; -- 2.37.1 From bbb860e3dca8fa78d78c055b932c62f03e92c50d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 29 Apr 2015 10:55:40 +0200 Subject: [PATCH 258/310] ggated send_thread(): Assert that we only send data for read requests Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index cd36dc853c2a..f6a4f653c084 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -907,6 +907,7 @@ send_thread(void *arg) g_gate_log(LOG_DEBUG, "Sent hdr packet."); g_gate_swap2h_hdr(&req->r_hdr); if (req->r_data != NULL) { + assert(req->r_cmd == GGATE_CMD_READ); data = g_gate_send(fd, req->r_data, req->r_length, 0); if (data != (ssize_t)req->r_length) { g_gate_xlog("Error while sending data: %s.", -- 2.37.1 From 19ee75cbcd604c5cdc25f733800c30ecb4a426c0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 4 May 2015 18:00:04 +0200 Subject: [PATCH 259/310] ggated: Open the listening socket CLOEXEC Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index f6a4f653c084..50ce8fd3d5d4 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1133,7 +1133,7 @@ main(int argc, char *argv[]) signal(SIGCHLD, SIG_IGN); signal(SIGPIPE, SIG_IGN); - sfd = socket(AF_INET, SOCK_STREAM, 0); + sfd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0); if (sfd == -1) g_gate_xlog("Cannot open stream socket: %s.", strerror(errno)); bzero(&serv, sizeof(serv)); -- 2.37.1 From 27f062c2a98b6b2e1f2eb18d36fbeb34abcb3d02 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 4 May 2015 18:31:46 +0200 Subject: [PATCH 260/310] ggated: Fix another socket leak Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 50ce8fd3d5d4..6631a23d8b93 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -344,6 +344,11 @@ exports_check(struct ggd_export *ex, struct g_gate_cinit *cinit, return (EPERM); } } + if (conn->c_diskfd != -1) { + g_gate_log(LOG_DEBUG, "Requested file %s is already open: %d", + ex->e_path, conn->c_diskfd); + return(0); + } if ((conn->c_flags & GGATE_FLAG_RDONLY) != 0) flags = O_RDONLY; else if ((conn->c_flags & GGATE_FLAG_WRONLY) != 0) -- 2.37.1 From 317d001131f4e8c9c9e1d1865746ba42b48d4dfa Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 30 Apr 2015 11:52:06 +0200 Subject: [PATCH 261/310] ggated recv_thread(): In case of read-only files, only accept read commands Accepting write commands etc. is not a security problem because the file descriptor isn't writeable anyway, but accepting requests other than reads could hide client bugs. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 6631a23d8b93..fef9e831a4ff 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -694,6 +694,12 @@ recv_thread(void *arg) "not fit sector size."); } + if ((conn->c_flags & GGATE_FLAG_RDONLY) != 0 + && req->r_cmd != GGATE_CMD_READ) { + g_gate_xlog("%s request received for read-only file", + g_gate_cmd2str(req->r_cmd)); + } + /* * Limit the amount of memory we allocate on behalf of * the client. MAXPHYS is the hard limit in ggatec, -- 2.37.1 From 582203e675d73e8ecbfe672e758b778cb47a872f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 3 May 2015 14:02:02 +0200 Subject: [PATCH 262/310] ggatec: Add log-to-file support Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 5 ++++- sbin/ggate/shared/ggate.c | 26 +++++++++++++++++++++----- sbin/ggate/shared/ggate.h | 1 + 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index f6ba77a9d034..86a8f147c562 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -662,7 +662,7 @@ main(int argc, char *argv[]) int ch; char *p; - ch = getopt(argc, argv, "fno:p:q:R:S:s:t:T:u:v"); + ch = getopt(argc, argv, "fl:no:p:q:R:S:s:t:T:u:v"); if (ch == -1) break; switch (ch) { @@ -671,6 +671,9 @@ main(int argc, char *argv[]) usage(); force = 1; break; + case 'l': + g_gate_open_log(optarg); + break; case 'n': if (action != CREATE && action != RESCUE) usage(); diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index d6be4948d9c6..084db7304e53 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -28,6 +28,7 @@ * $FreeBSD$ */ +#define _WITH_DPRINTF #include #include #include @@ -61,13 +62,23 @@ int g_gate_devfd = -1; int g_gate_verbose = 0; +static int g_gate_logfd = -1; +void +g_gate_open_log(const char *logfile) +{ + + g_gate_logfd = open(logfile, O_CREAT | O_WRONLY | O_APPEND, S_IWUSR |S_IRUSR); + if (g_gate_logfd == -1) { + g_gate_xlog("Failed to open %s: %s", logfile, strerror(errno)); + } +} void g_gate_vlog(int priority, const char *message, va_list ap) { - if (g_gate_verbose) { + if (g_gate_verbose || g_gate_logfd != -1) { const char *prefix; switch (priority) { @@ -89,10 +100,15 @@ g_gate_vlog(int priority, const char *message, va_list ap) default: prefix = "unknown"; } - - printf("%s: ", prefix); - vprintf(message, ap); - printf("\n"); + if (g_gate_logfd == -1) { + printf("%s: ", prefix); + vprintf(message, ap); + printf("\n"); + } else if (g_gate_verbose || priority != LOG_DEBUG) { + dprintf(g_gate_logfd, "%s: ", prefix); + vdprintf(g_gate_logfd, message, ap); + dprintf(g_gate_logfd, "\n"); + } } else { if (priority != LOG_DEBUG) vsyslog(priority, message, ap); diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index f2b602670088..7b72e6e192f2 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -97,6 +97,7 @@ struct g_gate_hdr { uint16_t gh_error; /* error value (0 if ok) */ } __packed; +void g_gate_open_log(const char *logfile); void g_gate_vlog(int priority, const char *message, va_list ap); void g_gate_log(int priority, const char *message, ...); void g_gate_xvlog(const char *message, va_list ap) __dead2; -- 2.37.1 From 0abf9d8487bdaddf2e02ad1beef0fdbbc91155a5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 30 Apr 2015 13:52:39 +0200 Subject: [PATCH 263/310] ggate[cd]: Add Jail and Capsicum support The capsicum support for ggatec is incomplete and only enabled if the -c flag is used as it currently prevents ggatec from reconnecting which is very inconvenient. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.8 | 8 +++ sbin/ggate/ggatec/ggatec.c | 37 ++++++++-- sbin/ggate/ggated/ggated.c | 5 ++ sbin/ggate/shared/ggate.c | 137 +++++++++++++++++++++++++++++++++++++ sbin/ggate/shared/ggate.h | 2 + 5 files changed, 182 insertions(+), 7 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.8 b/sbin/ggate/ggatec/ggatec.8 index ba3c9ecb6f0b..f4bf7cf43902 100644 --- a/sbin/ggate/ggatec/ggatec.8 +++ b/sbin/ggate/ggatec/ggatec.8 @@ -33,6 +33,7 @@ .Sh SYNOPSIS .Nm .Cm create +.Op Fl c .Op Fl n .Op Fl v .Op Fl o Cm ro | wo | rw @@ -48,6 +49,7 @@ .Ar path .Nm .Cm rescue +.Op Fl c .Op Fl n .Op Fl v .Op Fl o Cm ro | wo | rw @@ -104,6 +106,12 @@ providers. .Pp Available options: .Bl -tag -width ".Fl s Cm ro | wo | rw" +.It Fl c +Enter capsicum sandbox. +Currently this prevents +.Nm ggatec +from reconnecting which is somewhat inconvenient. +The flag will go away once this is fixed. .It Fl f Forcibly destroy .Nm ggate diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 86a8f147c562..0b8345068150 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -53,6 +53,7 @@ #include #include #include +#include #include #include "ggate.h" @@ -62,6 +63,8 @@ static enum { UNSET, CREATE, DESTROY, LIST, RESCUE } action = UNSET; static const char *path = NULL; static const char *host = NULL; +static in_addr_t host_ip; +static const char *logfile = NULL; static int unit = G_GATE_UNIT_AUTO; static unsigned flags = 0; static int force = 0; @@ -77,6 +80,7 @@ static uint32_t token; static pthread_t sendtd, recvtd; static int reconnect; static int initialbuffersize = 131072; +static int drop_capabilities = 0; static void usage(void) @@ -389,7 +393,7 @@ handshake(int dir) */ bzero(&serv, sizeof(serv)); serv.sin_family = AF_INET; - serv.sin_addr.s_addr = g_gate_str2ip(host); + serv.sin_addr.s_addr = host_ip; if (serv.sin_addr.s_addr == INADDR_NONE) { g_gate_log(LOG_DEBUG, "Invalid IP/host name: %s.", host); return (-1); @@ -497,7 +501,7 @@ static void mydaemon(void) { - if (g_gate_verbose > 0) + if (logfile == NULL && g_gate_verbose > 0) return; if (daemon(0, 0) == 0) return; @@ -562,6 +566,10 @@ g_gatec_loop(void) signal(SIGUSR1, signop); for (;;) { g_gatec_start(); + + if (cap_sandboxed()) + g_gate_xlog("Got disconnected while being sandboxed."); + g_gate_log(LOG_NOTICE, "Disconnected [%s %s]. Connecting...", host, path); while (!g_gatec_connect()) { @@ -584,9 +592,6 @@ g_gatec_create(void) if (!g_gatec_connect()) g_gate_xlog("Cannot connect: %s.", strerror(errno)); - /* - * Ok, got both sockets, time to create provider. - */ memset(&ggioc, 0, sizeof(ggioc)); ggioc.gctl_version = G_GATE_VERSION; ggioc.gctl_mediasize = mediasize; @@ -609,6 +614,9 @@ g_gatec_create(void) } unit = ggioc.gctl_unit; + if (drop_capabilities) + g_gate_drop_capabilities(sendfd, recvfd); + mydaemon(); g_gatec_loop(); } @@ -621,6 +629,9 @@ g_gatec_rescue(void) if (!g_gatec_connect()) g_gate_xlog("Cannot connect: %s.", strerror(errno)); + if (drop_capabilities) + g_gate_drop_capabilities(sendfd, recvfd); + ggioc.gctl_version = G_GATE_VERSION; ggioc.gctl_unit = unit; ggioc.gctl_seq = 0; @@ -662,17 +673,21 @@ main(int argc, char *argv[]) int ch; char *p; - ch = getopt(argc, argv, "fl:no:p:q:R:S:s:t:T:u:v"); + ch = getopt(argc, argv, "cfl:no:p:q:R:S:s:t:T:u:v"); if (ch == -1) break; switch (ch) { + case 'c': + drop_capabilities = 1; + force = 1; + break; case 'f': if (action != DESTROY) usage(); force = 1; break; case 'l': - g_gate_open_log(optarg); + logfile = optarg; break; case 'n': if (action != CREATE && action != RESCUE) @@ -786,7 +801,11 @@ main(int argc, char *argv[]) g_gate_load_module(); g_gate_open_device(); host = argv[0]; + host_ip = g_gate_str2ip(host); path = argv[1]; + if (logfile != NULL) + g_gate_open_log(logfile); + g_gate_drop_privs("hast", host_ip); g_gatec_create(); break; case DESTROY: @@ -810,7 +829,11 @@ main(int argc, char *argv[]) } g_gate_open_device(); host = argv[0]; + host_ip = g_gate_str2ip(host); path = argv[1]; + if (logfile != NULL) + g_gate_open_log(logfile); + g_gate_drop_privs("hast", host_ip); g_gatec_rescue(); break; case UNSET: diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index fef9e831a4ff..d69d5691acb0 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -560,6 +560,11 @@ connection_launch(struct ggd_connection *conn) } g_gate_log(LOG_DEBUG, "Process created [%s].", conn->c_path); + if (getuid() == 0) + g_gate_drop_privs("hast", bindaddr); + + g_gate_drop_capabilities(conn->c_sendfd, conn->c_recvfd); + /* * Create condition variables and mutexes for in-queue and out-queue * synchronization. diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index 084db7304e53..4c0d3886f463 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -55,6 +55,11 @@ #include #include #include +#include +#include +#include +#include +#include #include #include "ggate.h" @@ -443,3 +448,135 @@ g_gate_cmd2str(int cmd) return ("unknown (invalid?) GGATE command"); } + +/* + * The functions below are based on drop_privs() from ../../hastd/subr.c + * + * Changes: + * - HAST_USER replaced with ggate_user option + * - pjdlog_* replaced with g_gate_xlog(). + * - Don't fall back to chroot if jailing fails. + */ +#define PJDLOG_VERIFY assert +void +g_gate_drop_privs(const char *ggate_user, in_addr_t jail_address) +{ + char jailhost[32]; + struct jail jailst; + struct passwd *pw; + uid_t ruid, euid, suid; + gid_t rgid, egid, sgid; + gid_t gidset[1]; + struct in_addr jail_ip; + /* + * According to getpwnam(3) we have to clear errno before calling the + * function to be able to distinguish between an error and missing + * entry (with is not treated as error by getpwnam(3)). + */ + errno = 0; + pw = getpwnam(ggate_user); + if (pw == NULL) { + if (errno != 0) { + g_gate_xlog("Unable to find info about '%s' user", + ggate_user); + } else { + g_gate_xlog("'%s' user doesn't exist.", ggate_user); + } + } + + jail_ip.s_addr = jail_address; + + bzero(&jailst, sizeof(jailst)); + jailst.version = JAIL_API_VERSION; + jailst.path = pw->pw_dir; + (void)snprintf(jailhost, sizeof(jailhost), "%s-jail", getprogname()); + jailst.hostname = jailhost; + jailst.jailname = NULL; + jailst.ip4s = 1; + jailst.ip4 = &jail_ip; + jailst.ip6s = 0; + jailst.ip6 = NULL; + if (jail(&jailst) == -1) { + g_gate_xlog("Unable to jail process in directory %s", pw->pw_dir); + } + PJDLOG_VERIFY(chdir("/") == 0); + gidset[0] = pw->pw_gid; + if (setgroups(1, gidset) == -1) { + g_gate_xlog("Unable to set groups to gid %u", + (unsigned int)pw->pw_gid); + } + if (setgid(pw->pw_gid) == -1) { + g_gate_xlog("Unable to set gid to %u", + (unsigned int)pw->pw_gid); + } + if (setuid(pw->pw_uid) == -1) { + g_gate_xlog("Unable to set uid to %u", + (unsigned int)pw->pw_uid); + } + + /* + * Better be sure that everything succeeded. + */ + PJDLOG_VERIFY(getresuid(&ruid, &euid, &suid) == 0); + PJDLOG_VERIFY(ruid == pw->pw_uid); + PJDLOG_VERIFY(euid == pw->pw_uid); + PJDLOG_VERIFY(suid == pw->pw_uid); + PJDLOG_VERIFY(getresgid(&rgid, &egid, &sgid) == 0); + PJDLOG_VERIFY(rgid == pw->pw_gid); + PJDLOG_VERIFY(egid == pw->pw_gid); + PJDLOG_VERIFY(sgid == pw->pw_gid); + PJDLOG_VERIFY(getgroups(0, NULL) == 1); + PJDLOG_VERIFY(getgroups(1, gidset) == 1); + PJDLOG_VERIFY(gidset[0] == pw->pw_gid); + + g_gate_log(LOG_DEBUG, "Privileges successfully dropped using " + "jail+setgid+setuid."); +} + +int +g_gate_drop_capabilities(int sendfd, int recvfd) +{ + cap_rights_t rights; + static const unsigned long ggatecmds[] = { + G_GATE_CMD_START, + G_GATE_CMD_DONE, + G_GATE_CMD_CANCEL, + }; + + if (cap_enter() != 0) { + g_gate_xlog("Failed to sandbox using capsicum"); + } + + cap_rights_init(&rights, CAP_PREAD, CAP_PWRITE); + if (cap_rights_limit(sendfd, &rights) == -1) { + g_gate_xlog("Unable to limit capability " + "rights on sendfd %d", sendfd); + } + if (cap_rights_limit(recvfd, &rights) == -1) { + g_gate_xlog("Unable to limit capability " + "rights on recvfd %d", recvfd); + } + + /* Only the client uses this. */ + if (g_gate_devfd != -1) { + cap_rights_init(&rights, CAP_IOCTL, CAP_PREAD, CAP_PWRITE); + if (cap_rights_limit(g_gate_devfd, &rights) == -1) { + g_gate_xlog("Unable to limit capability rights " + "to CAP_IOCTL on ggate descriptor"); + } + if (cap_ioctls_limit(g_gate_devfd, ggatecmds, + sizeof(ggatecmds) / sizeof(ggatecmds[0])) == -1) { + g_gate_xlog("Unable to limit allowed ggate ioctls"); + } + } + cap_rights_init(&rights, CAP_PWRITE); + if (g_gate_logfd != -1 && + cap_rights_limit(g_gate_logfd, &rights) == -1) { + g_gate_xlog("Unable to limit capability " + "rights on logfd %d", g_gate_logfd); + } + + g_gate_log(LOG_DEBUG, "Entered Capsicum sandbox"); + + return (0); +} diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 7b72e6e192f2..45ba2c9d15e8 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -117,6 +117,8 @@ void g_gate_list(int unit, int verbose); #endif in_addr_t g_gate_str2ip(const char *str); const char *g_gate_cmd2str(int cmd); +void g_gate_drop_privs(const char *ggate_user, in_addr_t jail_address); +int g_gate_drop_capabilities(int sendfd, int recvfd); /* * g_gate_swap2h_* - functions swap bytes to host byte order (from big endian). -- 2.37.1 From fbcdf684391076467d24d070299ce70773f9cc37 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 28 Apr 2015 13:02:25 +0200 Subject: [PATCH 264/310] Bump GGATE_VERSION due to FLUSH and DELETE support and various bug fixes Unpatched ggate[cd] versions may cause data corruption so we no longer want to speak to them. Obtained from: ElectroBSD --- sbin/ggate/shared/ggate.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 45ba2c9d15e8..2359b0af6ddf 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -42,7 +42,7 @@ #define G_GATE_TIMEOUT 0 #define GGATE_MAGIC "GEOM_GATE " -#define GGATE_VERSION 0 +#define GGATE_VERSION 1 #define GGATE_FLAG_RDONLY 0x0001 #define GGATE_FLAG_WRONLY 0x0002 -- 2.37.1 From 416b9d709c41892ad5d3e6ec29a87293310a46c4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 9 Aug 2015 15:20:48 +0200 Subject: [PATCH 265/310] ggate: Use dedicated users for ggatec and ggated Obtained from: ElectroBSD --- etc/group | 2 ++ etc/master.passwd | 2 ++ sbin/ggate/ggatec/ggatec.c | 4 ++-- sbin/ggate/ggated/ggated.c | 2 +- 4 files changed, 7 insertions(+), 3 deletions(-) diff --git a/etc/group b/etc/group index f48fef985e4d..27fc59dee2b9 100644 --- a/etc/group +++ b/etc/group @@ -35,6 +35,8 @@ ntpd:*:123: _ypldap:*:160: _tor:*:256: hast:*:845: +ggatec:*:846: +ggated:*:847: tests:*:977: nogroup:*:65533: nobody:*:65534: diff --git a/etc/master.passwd b/etc/master.passwd index 597aebe125dc..d1ad7c159d50 100644 --- a/etc/master.passwd +++ b/etc/master.passwd @@ -26,5 +26,7 @@ ntpd:*:123:123::0:0:NTP Daemon:/var/db/ntp:/usr/sbin/nologin _ypldap:*:160:160::0:0:YP LDAP unprivileged user:/var/empty:/usr/sbin/nologin _tor:*:256:256::0:0:Onion delivery agent:/nonexistent:/usr/sbin/nologin hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin +ggatec:*:846:846::0:0:ggatec unprivileged user:/var/empty:/usr/sbin/nologin +ggated:*:847:847::0:0:ggated unprivileged user:/var/empty:/usr/sbin/nologin tests:*:977:977::0:0:Unprivileged user for tests:/nonexistent:/usr/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 0b8345068150..839239eb744a 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -805,7 +805,7 @@ main(int argc, char *argv[]) path = argv[1]; if (logfile != NULL) g_gate_open_log(logfile); - g_gate_drop_privs("hast", host_ip); + g_gate_drop_privs("ggatec", host_ip); g_gatec_create(); break; case DESTROY: @@ -833,7 +833,7 @@ main(int argc, char *argv[]) path = argv[1]; if (logfile != NULL) g_gate_open_log(logfile); - g_gate_drop_privs("hast", host_ip); + g_gate_drop_privs("ggatec", host_ip); g_gatec_rescue(); break; case UNSET: diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index d69d5691acb0..810c183c1b49 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -561,7 +561,7 @@ connection_launch(struct ggd_connection *conn) g_gate_log(LOG_DEBUG, "Process created [%s].", conn->c_path); if (getuid() == 0) - g_gate_drop_privs("hast", bindaddr); + g_gate_drop_privs("ggated", bindaddr); g_gate_drop_capabilities(conn->c_sendfd, conn->c_recvfd); -- 2.37.1 From 51d7d788743c6489383ac31df0c4be943d3704fb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 13 Nov 2021 09:56:26 +0100 Subject: [PATCH 266/310] ggated: Close the pid file and directory after forking ... so we're allowed to go to jail without having to set kern.pwd_chroot_chdir_check_open_directories=0 again. See also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259770 Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 810c183c1b49..6db8a394303b 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -95,6 +95,7 @@ struct ggd_export { SLIST_ENTRY(ggd_export) e_next; }; +static struct pidfh *pfh; static const char *exports_file = GGATED_EXPORT_FILE; static int got_sighup = 0; static in_addr_t bindaddr; @@ -560,6 +561,9 @@ connection_launch(struct ggd_connection *conn) } g_gate_log(LOG_DEBUG, "Process created [%s].", conn->c_path); + if (pidfile_close(pfh) == -1) + g_gate_xlog("pidfile_close(): %s.", strerror(errno)); + if (getuid() == 0) g_gate_drop_privs("ggated", bindaddr); @@ -1071,7 +1075,6 @@ int main(int argc, char *argv[]) { const char *ggated_pidfile = _PATH_VARRUN "/ggated.pid"; - struct pidfh *pfh; struct sockaddr_in serv; struct sockaddr from; socklen_t fromlen; -- 2.37.1 From 8f994f2bac88916b189c4e1d371210a9950f3349 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 13 Nov 2021 10:45:08 +0100 Subject: [PATCH 267/310] ggated: Let connection_launch() close the accepting socket after forking There's no reason why the children should have access to it. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 6db8a394303b..7870f929cc66 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -547,7 +547,7 @@ connection_ready(struct ggd_connection *conn) } static void -connection_launch(struct ggd_connection *conn) +connection_launch(struct ggd_connection *conn, int sfd) { pthread_t td; int error, pid; @@ -563,6 +563,8 @@ connection_launch(struct ggd_connection *conn) if (pidfile_close(pfh) == -1) g_gate_xlog("pidfile_close(): %s.", strerror(errno)); + if (close(sfd) == -1) + g_gate_xlog("close(sfd): %s.", strerror(errno)); if (getuid() == 0) g_gate_drop_privs("ggated", bindaddr); @@ -955,7 +957,7 @@ log_connection(struct sockaddr *from) } static int -handshake(struct sockaddr *from, int sfd) +handshake(struct sockaddr *from, int sfd, int listen_fd) { struct g_gate_version ver; struct g_gate_cinit cinit; @@ -1058,7 +1060,7 @@ handshake(struct sockaddr *from, int sfd) } if (connection_ready(conn)) { - connection_launch(conn); + connection_launch(conn, listen_fd); connection_remove(conn); } return (1); @@ -1184,7 +1186,7 @@ main(int argc, char *argv[]) exports_get(); } - if (!handshake(&from, tmpsfd)) + if (!handshake(&from, tmpsfd, sfd)) close(tmpsfd); } close(sfd); -- 2.37.1 From ca7012c3d71b92cb4cd1be1fe7b9a79ce3aec9b5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Nov 2021 15:06:57 +0100 Subject: [PATCH 268/310] ggated: Add undocumented -j option to test jailing Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 7870f929cc66..7f8d6bf2adef 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1083,10 +1083,11 @@ main(int argc, char *argv[]) pid_t otherpid; int ch, sfd, tmpsfd; unsigned port; + int g_gate_jail_test = 0; bindaddr = htonl(INADDR_ANY); port = G_GATE_PORT; - while ((ch = getopt(argc, argv, "a:hnp:F:R:S:v")) != -1) { + while ((ch = getopt(argc, argv, "a:hnjp:F:R:S:v")) != -1) { switch (ch) { case 'a': bindaddr = g_gate_str2ip(optarg); @@ -1098,6 +1099,9 @@ main(int argc, char *argv[]) case 'F': ggated_pidfile = optarg; break; + case 'j': + g_gate_jail_test = 1; + break; case 'n': nagle = 0; break; @@ -1132,7 +1136,8 @@ main(int argc, char *argv[]) if (argv[0] != NULL) exports_file = argv[0]; - exports_get(); + if (!g_gate_jail_test) + exports_get(); pfh = pidfile_open(ggated_pidfile, 0600, &otherpid); if (pfh == NULL) { @@ -1173,6 +1178,13 @@ main(int argc, char *argv[]) signal(SIGHUP, huphandler); + if (g_gate_jail_test) { + pidfile_close(pfh); + g_gate_drop_privs("ggated", bindaddr); + pidfile_remove(pfh); + exit(EXIT_SUCCESS); + } + for (;;) { fromlen = sizeof(from); tmpsfd = accept(sfd, &from, &fromlen); -- 2.37.1 From 94b34146dd22fa3a893639e79d853cf186e856d1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Nov 2021 16:32:40 +0100 Subject: [PATCH 269/310] ggated: Add undocumented -J flag to disable jailing Should work around: fk@r500 ~ $sudo ggated -v -j info: Reading exports file (/etc/gg.exports). debug: Added 127.0.0.1/32 /dev/zvol/r500/ggated/t520.eli RW to exports list. info: Exporting 1 object(s). info: Listen on port: 3080. error: Unable to jail process in directory /var/empty error: Exiting. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 5 ++++- sbin/ggate/shared/ggate.c | 3 ++- sbin/ggate/shared/ggate.h | 1 + 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 7f8d6bf2adef..9203ead8d323 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1087,7 +1087,7 @@ main(int argc, char *argv[]) bindaddr = htonl(INADDR_ANY); port = G_GATE_PORT; - while ((ch = getopt(argc, argv, "a:hnjp:F:R:S:v")) != -1) { + while ((ch = getopt(argc, argv, "a:hnjJp:F:R:S:v")) != -1) { switch (ch) { case 'a': bindaddr = g_gate_str2ip(optarg); @@ -1099,6 +1099,9 @@ main(int argc, char *argv[]) case 'F': ggated_pidfile = optarg; break; + case 'J': + g_gate_no_jailing = 1; + break; case 'j': g_gate_jail_test = 1; break; diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index 4c0d3886f463..05a2cdcd1f83 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -66,6 +66,7 @@ int g_gate_devfd = -1; +int g_gate_no_jailing = 0; int g_gate_verbose = 0; static int g_gate_logfd = -1; @@ -496,7 +497,7 @@ g_gate_drop_privs(const char *ggate_user, in_addr_t jail_address) jailst.ip4 = &jail_ip; jailst.ip6s = 0; jailst.ip6 = NULL; - if (jail(&jailst) == -1) { + if (!g_gate_no_jailing && jail(&jailst) == -1) { g_gate_xlog("Unable to jail process in directory %s", pw->pw_dir); } PJDLOG_VERIFY(chdir("/") == 0); diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 2359b0af6ddf..e3af84e437ea 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -61,6 +61,7 @@ #define GGATE_CMD_DELETE 4 extern int g_gate_devfd; +extern int g_gate_no_jailing; extern int g_gate_verbose; extern int nagle; -- 2.37.1 From 965d19a73ff6c67f434fea1077d6bce43a67eb67 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 19 Apr 2015 22:58:49 +0200 Subject: [PATCH 270/310] ggated: Default to listening to 127.0.0.1 only Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 9203ead8d323..35a097954203 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1085,7 +1085,7 @@ main(int argc, char *argv[]) unsigned port; int g_gate_jail_test = 0; - bindaddr = htonl(INADDR_ANY); + bindaddr = g_gate_str2ip("127.0.0.1"); port = G_GATE_PORT; while ((ch = getopt(argc, argv, "a:hnjJp:F:R:S:v")) != -1) { switch (ch) { -- 2.37.1 From 19890bfbed85ce3e93604aa078d510206605f96c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 29 Nov 2021 08:50:52 +0100 Subject: [PATCH 271/310] release/scripts/strip-freebsd.sh: Ditch *.ucode files *.ucode files were added for iwlwifi in f4c129f5fb1669b28 but haven't been MFC'd to stable/12 (yet). Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 556b2ba4e365..fb0438300078 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -83,6 +83,9 @@ get_usb_firmware_files() { # by non-free dependencies get_unfree_files() { find sys/ -name "*.uu" + # *.ucode files were added for iwlwifi in f4c129f5fb1669b28 + # which hasn't been MFC'd to stable/12 (yet). + find sys/ -name "*.ucode" get_dev_ice_files get_dev_imx_files get_dev_qatfw_files -- 2.37.1 From 3f76f0e93a2392ea61b6a0d5afead1708d2e12c1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 12:50:24 +0100 Subject: [PATCH 272/310] stand/lua/gfx-beastie.lua: Replace fork with toilet brush Obtained from: ElectroBSD --- stand/lua/gfx-beastie.lua | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/stand/lua/gfx-beastie.lua b/stand/lua/gfx-beastie.lua index f88f8c8dd306..dc69c0413120 100644 --- a/stand/lua/gfx-beastie.lua +++ b/stand/lua/gfx-beastie.lua @@ -41,9 +41,9 @@ return { " (_.) _ ) /", " `.___/` /", " `-----' /", - " \027[33m<----.\027[31m __ / __ \\", - " \027[33m<----|====\027[31mO)))\027[33m==\027[31m) \\) /\027[33m====|", - " \027[33m<----'\027[31m `--' `.__,' \\", + " \027[33m######\027[31m __ / __ \\", + " \027[33m######====\027[31mO)))\027[33m==\027[31m) \\) /\027[33m====|", + " \027[33m######\027[31m `--' `.__,' \\", " | |", " \\ / /\\", " \027[36m______\027[31m( (_ / \\______/", -- 2.37.1 From 7b6717cbedcbb35dbde14ee00f83f42fe286e21c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 12:51:25 +0100 Subject: [PATCH 273/310] stand/lua/gfx-beastiebw.lua: Replace fork with toilet brush Obtained from: ElectroBSD --- stand/lua/gfx-beastiebw.lua | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/stand/lua/gfx-beastiebw.lua b/stand/lua/gfx-beastiebw.lua index 5895b2436542..8ae8599c3b57 100644 --- a/stand/lua/gfx-beastiebw.lua +++ b/stand/lua/gfx-beastiebw.lua @@ -41,9 +41,9 @@ return { " (_.) _ ) /", " `.___/` /", " `-----' /", - " <----. __ / __ \\", - " <----|====O)))==) \\) /====|", - " <----' `--' `.__,' \\", + " ###### __ / __ \\", + " ######====O)))==) \\) /====|", + " ###### `--' `.__,' \\", " | |", " \\ / /\\", " ______( (_ / \\______/", -- 2.37.1 From 11f3e85388493f91a7130e551848ebfdd90a3a7c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 13:03:37 +0100 Subject: [PATCH 274/310] stand/defaults/loader.conf: Set beastie_disable="NO" Obtained from: ElectroBSD --- stand/defaults/loader.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/defaults/loader.conf b/stand/defaults/loader.conf index 24a47e75c39a..74cc6a222e57 100644 --- a/stand/defaults/loader.conf +++ b/stand/defaults/loader.conf @@ -94,7 +94,7 @@ audit_event_type="etc_security_audit_event" #bootlock_password="" # Prevent booting (see check-password.4th(8)) #geom_eli_passphrase_prompt="NO" # Prompt for geli(8) passphrase to mount root bootenv_autolist="YES" # Auto populate the list of ZFS Boot Environments -#beastie_disable="NO" # Turn the beastie boot menu on and off +beastie_disable="NO" # Turn the beastie boot menu on and off efi_max_resolution="1x1" # Set the max resolution for EFI loader to use: # 480p, 720p, 1080p, 2160p/4k, 5k, or specify # WidthxHeight (e.g. 1920x1080) -- 2.37.1 From d2251f06cd16e806fe38eea6cad2387e81fc738b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 13:50:06 +0100 Subject: [PATCH 275/310] stand/lua/drawer.lua: Welcome users to ElectroBSD instead of FreeBSD Obtained from: ElectroBSD --- stand/lua/drawer.lua | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/lua/drawer.lua b/stand/lua/drawer.lua index 6f6e2fe1b3e8..cfe197a774b1 100644 --- a/stand/lua/drawer.lua +++ b/stand/lua/drawer.lua @@ -261,7 +261,7 @@ local function drawbox() local y = menu_position.y - 1 local w = frame_size.w local menu_header = loader.getenv("loader_menu_title") or - "Welcome to FreeBSD" + "Welcome to ElectroBSD" local menu_header_align = loader.getenv("loader_menu_title_align") local menu_header_x -- 2.37.1 From d7c90451d26fd0153dfc39dc046fea59620f1579 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 12:57:20 +0100 Subject: [PATCH 276/310] stand/lua/drawer.lua: Change the OS name in the banner to ElectroBSD While at it, suggest to resist unlawful police activities (German). Obtained from: ElectroBSD --- stand/lua/drawer.lua | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/stand/lua/drawer.lua b/stand/lua/drawer.lua index cfe197a774b1..954ee13083f8 100644 --- a/stand/lua/drawer.lua +++ b/stand/lua/drawer.lua @@ -390,14 +390,17 @@ local function drawitem(func) loader.setenv("console", console) end +-- XXX: The variable should probably be renamed to 'electrobsd_brand' fbsd_brand = { -" ______ ____ _____ _____ ", -" | ____| | _ \\ / ____| __ \\ ", -" | |___ _ __ ___ ___ | |_) | (___ | | | |", -" | ___| '__/ _ \\/ _ \\| _ < \\___ \\| | | |", -" | | | | | __/ __/| |_) |____) | |__| |", -" | | | | | | || | | |", -" |_| |_| \\___|\\___||____/|_____/|_____/ " +" ______ _ _ ____ _____ _____", +" | ____| | | | | _ \\ / ____| __ \\", +" | |__ | | ___ ___| |_ _ __ ___ | |_) | (___ | | | |", +" | __| | |/ _ \\/ __| __| '__/ _ \\| _ < \\___ \\| | | |", +" | |____| | __/ (__| |_| | | (_) | |_) |____) | |__| |", +" |______|_|\\___|\\___|\\__|_| \\___/|____/|_____/|_____/", +" Polizei-Willkuer in Deinem Land? Das erfordert Widerstand!", +" Zu Risiken und Nebenwirkungen fragen Sie Ihren Anwalt oder", +" die Rote Hilfe." } none = {""} -- 2.37.1 From 0d437c5ef0741c57b8fd6c02b03d6027602077e1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 13:46:49 +0100 Subject: [PATCH 277/310] stand/lua/drawer.lua: Think of the children and default to the beastie logo ... instead of the 'sex toy'. Obtained from: ElectroBSD --- stand/lua/drawer.lua | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/stand/lua/drawer.lua b/stand/lua/drawer.lua index 954ee13083f8..cf3c684a8944 100644 --- a/stand/lua/drawer.lua +++ b/stand/lua/drawer.lua @@ -470,8 +470,8 @@ shift = default_shift -- Module exports drawer.default_brand = 'fbsd' -drawer.default_color_logodef = 'orb' -drawer.default_bw_logodef = 'orbbw' +drawer.default_color_logodef = 'beastie' +drawer.default_bw_logodef = 'beastiebw' -- For when things go terribly wrong; this def should be present here in the -- drawer module in case it's a filesystem issue. drawer.default_fallback_logodef = 'none' -- 2.37.1 From 9f8413dc8057d557d92ae4c88eb325106de08d8f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 5 Dec 2021 12:09:15 +0100 Subject: [PATCH 278/310] share/dtrace/flowtrace: Remove probes that are no longer valid Obtained from: ElectroBSD --- share/dtrace/flowtrace | 2 -- 1 file changed, 2 deletions(-) diff --git a/share/dtrace/flowtrace b/share/dtrace/flowtrace index 9e890f61a311..805a54592d43 100755 --- a/share/dtrace/flowtrace +++ b/share/dtrace/flowtrace @@ -62,7 +62,6 @@ fbt::$$1:entry /* ... except for the children of these functions. */ fbt::hpet_intr:return, -fbt::lapic_handle_intr:return, fbt::termcn_cnputc:return, fbt::printf:return, fbt::cv_signal:return, @@ -86,7 +85,6 @@ fbt::: } fbt::hpet_intr:entry, -fbt::lapic_handle_intr:entry, fbt::termcn_cnputc:entry, fbt::printf:entry, fbt::cv_signal:entry, -- 2.37.1 From 772e86b842a65d789876fd29e929e8c68585e37f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 3 Dec 2021 10:23:04 +0100 Subject: [PATCH 279/310] geli: Include sizes when the media size doesn't match ... the one of the backup file that is supposed to be restored. Use casts to make sure this compiles on both amd64 and i386. Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index 7e6e6a360387..1de864257132 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -1801,8 +1801,11 @@ eli_restore(struct gctl_req *req) if (gctl_get_int(req, "force")) { md.md_provsize = mediasize; } else { - gctl_error(req, "Provider size mismatch: " - "wrong backup file?"); + gctl_error(req, "Provider size mismatch for %s. " + "Provider size %llu, media size %llu. " + "Wrong backup file?", + prov, (unsigned long long)md.md_provsize, + (unsigned long long)mediasize); return; } } -- 2.37.1 From 2a647560ef349f1ee7211b4512397324b6065ba0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 5 Dec 2021 12:17:32 +0100 Subject: [PATCH 280/310] sys/geom/part: Ignore more errors if kern.geom.part.check_integrity is set to 0 ... and log more details about detected corruption in general. Use a bunch of ugly casts so it can be build on i386 as well. This commit was useful to recover from GPT corruption that looked like this: GEOM: hdr_lba_end (7814037127) < hdr->hdr_lba_start (40) or hrdr_lba_end >= last (7814035054) for ada1. GEOM: Reading sector 4000787029504 of size 512 from ada1. GEOM: ada1: the secondary GPT table is corrupt or invalid. GEOM: ada1: using the primary only -- recovery suggested. GEOM_PART: integrity check failed (ada1, GPT) Without this commit the partion table was complete invisble. With this commit and kern.geom.part.check_integrity=0 a corrupt GPT table was visible and looked like this: => 40 7814037088 ada1 GPT (3.6T) [CORRUPT] 40 512 1 freebsd-boot (256K) 552 1496 - free - (748K) 2048 409600 2 freebsd-zfs (200M) 411648 20971520 3 freebsd-zfs (10G) 21383168 8388608 4 freebsd-swap (4.0G) 29771776 7784263680 5 freebsd-zfs (3.6T) 7814035456 1672 - free - (836K) After deleting partition 5 and recreating it, all the data on it was available again: => 40 7814034982 ada1 GPT (3.6T) [CORRUPT] 40 512 1 freebsd-boot (256K) 552 1496 - free - (748K) 2048 409600 2 freebsd-zfs (200M) 411648 20971520 3 freebsd-zfs (10G) 21383168 8388608 4 freebsd-swap (4.0G) 29771776 7784263240 5 freebsd-zfs (3.6T) 7814035016 6 - free - (3.0K) After the next boot the [CORRUPT] marker was gone as well: => 40 7814034982 ada1 GPT (3.6T) 40 512 1 freebsd-boot (256K) 552 1496 - free - (748K) 2048 409600 2 freebsd-zfs (200M) 411648 20971520 3 freebsd-zfs (10G) 21383168 8388608 4 freebsd-swap (4.0G) 29771776 7784263240 5 freebsd-zfs (3.6T) 7814035016 6 - free - (3.0K) Unfortunately it's unclear how the GPT data got corrupted in the first place. Obtained from: ElectroBSD --- sys/geom/part/g_part_gpt.c | 132 +++++++++++++++++++++++++++++-------- 1 file changed, 103 insertions(+), 29 deletions(-) diff --git a/sys/geom/part/g_part_gpt.c b/sys/geom/part/g_part_gpt.c index 26274c6ae43f..10bfcb170612 100644 --- a/sys/geom/part/g_part_gpt.c +++ b/sys/geom/part/g_part_gpt.c @@ -457,66 +457,140 @@ gpt_read_hdr(struct g_part_gpt_table *table, struct g_consumer *cp, table->lba[elt] = 1; buf = g_read_data(cp, table->lba[elt] * pp->sectorsize, pp->sectorsize, &error); - if (buf == NULL) + if (buf == NULL) { + printf("GEOM: Reading sector %llu of size %d from %s.\n", + (unsigned long long)table->lba[elt] * pp->sectorsize, + pp->sectorsize, pp->name); return (NULL); + } hdr = NULL; if (memcmp(buf->hdr_sig, GPT_HDR_SIG, sizeof(buf->hdr_sig)) != 0) goto fail; table->state[elt] = GPT_STATE_CORRUPT; sz = le32toh(buf->hdr_size); - if (sz < 92 || sz > pp->sectorsize) - goto fail; - + if (sz < 92 || sz > pp->sectorsize) { + printf("GEOM: Strange sector size %d for %s.\n", + sz, pp->name); + if (geom_part_check_integrity) + goto fail; + } hdr = g_malloc(sz, M_WAITOK | M_ZERO); bcopy(buf, hdr, sz); hdr->hdr_size = sz; crc = le32toh(buf->hdr_crc_self); buf->hdr_crc_self = 0; - if (crc32(buf, sz) != crc) - goto fail; + if (crc32(buf, sz) != crc) { + printf("GEOM: %s: Invalid crc32 detected.\n", + pp->name); + if (geom_part_check_integrity) + goto fail; + } hdr->hdr_crc_self = crc; table->state[elt] = GPT_STATE_INVALID; hdr->hdr_revision = le32toh(buf->hdr_revision); - if (hdr->hdr_revision < GPT_HDR_REVISION) - goto fail; + if (hdr->hdr_revision < GPT_HDR_REVISION) { + printf("GEOM: Strange header revision %d for %s.\n", + hdr->hdr_revision, pp->name); + if (geom_part_check_integrity) + goto fail; + } hdr->hdr_lba_self = le64toh(buf->hdr_lba_self); - if (hdr->hdr_lba_self != table->lba[elt]) - goto fail; + if (hdr->hdr_lba_self != table->lba[elt]) { + printf("GEOM: hdr_lba_self (%llu) != table->lba[elt] " + "(%llu) for %s.\n", + (unsigned long long)hdr->hdr_lba_self, + (unsigned long long)table->lba[elt], pp->name); + if (geom_part_check_integrity) + goto fail; + } hdr->hdr_lba_alt = le64toh(buf->hdr_lba_alt); - if (hdr->hdr_lba_alt == hdr->hdr_lba_self) - goto fail; - if (hdr->hdr_lba_alt > last && geom_part_check_integrity) - goto fail; - + if (hdr->hdr_lba_alt == hdr->hdr_lba_self) { + printf("GEOM: hdr_lba_alt (%llu) == " + "hdr_lba_self (%llu) for %s.\n", + (unsigned long long)hdr->hdr_lba_alt, + (unsigned long long)hdr->hdr_lba_self, pp->name); + if (geom_part_check_integrity) + goto fail; + } + if (hdr->hdr_lba_alt > last && geom_part_check_integrity) { + printf("GEOM: hdr_lba_alt (%llu) > last (%llu) for %s.\n", + (unsigned long long)hdr->hdr_lba_alt, + (unsigned long long)last, pp->name); + if (geom_part_check_integrity) + goto fail; + } /* Check the managed area. */ hdr->hdr_lba_start = le64toh(buf->hdr_lba_start); - if (hdr->hdr_lba_start < 2 || hdr->hdr_lba_start >= last) - goto fail; + if (hdr->hdr_lba_start < 2 || hdr->hdr_lba_start >= last) { + printf("GEOM: hdr_lba_start (%llu) < 2 or >= last (%llu) for %s.\n", + (unsigned long long)hdr->hdr_lba_start, + (unsigned long long)last, pp->name); + if (geom_part_check_integrity) + goto fail; + } hdr->hdr_lba_end = le64toh(buf->hdr_lba_end); - if (hdr->hdr_lba_end < hdr->hdr_lba_start || hdr->hdr_lba_end >= last) - goto fail; + if (hdr->hdr_lba_end < hdr->hdr_lba_start || hdr->hdr_lba_end >= last) { + printf("GEOM: hdr_lba_end (%llu) < hdr->hdr_lba_start (%llu) or " + "hrdr_lba_end >= last (%llu) for %s.\n", + (unsigned long long)hdr->hdr_lba_end, + (unsigned long long)hdr->hdr_lba_start, + (unsigned long long)last, pp->name); + if (geom_part_check_integrity) + goto fail; + } /* Check the table location and size of the table. */ hdr->hdr_entries = le32toh(buf->hdr_entries); hdr->hdr_entsz = le32toh(buf->hdr_entsz); if (hdr->hdr_entries == 0 || hdr->hdr_entsz < 128 || - (hdr->hdr_entsz & 7) != 0) - goto fail; + (hdr->hdr_entsz & 7) != 0) { + printf("GEOM: hdr_entries (%u) == 0 or " + "hdr_entsz (%u) < 128 or " + "hdr_entsz & 7) != 0 for %s", + hdr->hdr_entries, hdr->hdr_entsz, pp->name); + if (geom_part_check_integrity) + goto fail; + } hdr->hdr_lba_table = le64toh(buf->hdr_lba_table); - if (hdr->hdr_lba_table < 2 || hdr->hdr_lba_table >= last) - goto fail; + if (hdr->hdr_lba_table < 2 || hdr->hdr_lba_table >= last) { + printf("GEOM: hdr_lba_table (%llu) < 2 or >= last (%llu)" + "for %s.\n", (unsigned long long)hdr->hdr_lba_table, + (unsigned long long)last, pp->name); + if (geom_part_check_integrity) + goto fail; + } if (hdr->hdr_lba_table >= hdr->hdr_lba_start && - hdr->hdr_lba_table <= hdr->hdr_lba_end) - goto fail; + hdr->hdr_lba_table <= hdr->hdr_lba_end) { + printf("GEOM: hdr_lba_table (%llu) >= hdr_lba_start (%llu) " + "or hdr_lba_table <= hdr_lba_end(%llu) for %s.\n", + (unsigned long long)hdr->hdr_lba_table, + (unsigned long long)hdr->hdr_lba_start, + (unsigned long long)hdr->hdr_lba_end, + pp->name); + if (geom_part_check_integrity) + goto fail; + } lba = hdr->hdr_lba_table + howmany(hdr->hdr_entries * hdr->hdr_entsz, pp->sectorsize) - 1; - if (lba >= last) - goto fail; - if (lba >= hdr->hdr_lba_start && lba <= hdr->hdr_lba_end) - goto fail; + if (lba >= last) { + printf("GEOM: lba (%llu) >= last (%llu) for %s.\n", + (unsigned long long)lba, (unsigned long long)last, + pp->name); + if (geom_part_check_integrity) + goto fail; + } + if (lba >= hdr->hdr_lba_start && lba <= hdr->hdr_lba_end) { + printf("GEOM: lba (%llu) >= hdr_lba_start (%llu) " + "&& lba <= hdr_lba_end (%llu) for %s.\n", + (unsigned long long)lba, + (unsigned long long)hdr->hdr_lba_start, + (unsigned long long)hdr->hdr_lba_end, pp->name); + if (geom_part_check_integrity) + goto fail; + } table->state[elt] = GPT_STATE_OK; le_uuid_dec(&buf->hdr_uuid, &hdr->hdr_uuid); -- 2.37.1 From e68aec1d558ce2cd7429d123d3e6a6716cc04c4f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 10 Dec 2021 18:45:27 +0100 Subject: [PATCH 281/310] release/amd64/make-memstick.sh: Produce sha256 for the mkimg input files and keep the partition for inspection Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 46fc90474574..95db57b90f2c 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -74,6 +74,10 @@ if [ $? -ne 0 ]; then echo "mkimg failed" exit 1 fi -rm ${espfilename} -rm ${2}.part +sha256 ${1}/boot/mbr +sha256 ${espfilename} +sha256 ${2}.part +#rm ${espfilename} +#rm ${2}.part + -- 2.37.1 From ced5e6e689c27908a62fe5d380f3ada6d82fc067 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 11 Dec 2021 11:36:58 +0100 Subject: [PATCH 282/310] release/amd64/make-memstick.sh: Generate sha256 for mtree spec Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 95db57b90f2c..2a75e5bfb78d 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -43,6 +43,8 @@ mtree -c -k time -p "${1}" | mtree -C -k all | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 +sha256 "${2}.mtree" + #define DEBUG_FS_MAKEFS 0x00000400 #define DEBUG_FS_CREATE_IMAGE 0x00001000 #define DEBUG_FS_WRITE_FILE 0x00080000 -- 2.37.1 From a53b7deefe5efe89b7606cd7644766e31d657457 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 11 Dec 2021 12:43:44 +0100 Subject: [PATCH 283/310] makefs: Let apply_specentry() use stampst if set Obtained from: ElectroBSD --- usr.sbin/makefs/walk.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/usr.sbin/makefs/walk.c b/usr.sbin/makefs/walk.c index 79b68d1d3e19..5978f88dc8d1 100644 --- a/usr.sbin/makefs/walk.c +++ b/usr.sbin/makefs/walk.c @@ -522,11 +522,17 @@ apply_specentry(const char *dir, NODE *specnode, fsnode *dirnode) (long)specnode->st_mtimespec.tv_sec); dirnode->inode->st.st_mtime = specnode->st_mtimespec.tv_sec; dirnode->inode->st.st_atime = specnode->st_mtimespec.tv_sec; - dirnode->inode->st.st_ctime = start_time.tv_sec; + if (stampst.st_ino != 0) + dirnode->inode->st.st_ctime = stampst.st_ctime; + else + dirnode->inode->st.st_ctime = start_time.tv_sec; #if HAVE_STRUCT_STAT_ST_MTIMENSEC dirnode->inode->st.st_mtimensec = specnode->st_mtimespec.tv_nsec; dirnode->inode->st.st_atimensec = specnode->st_mtimespec.tv_nsec; - dirnode->inode->st.st_ctimensec = start_time.tv_nsec; + if (stampst.st_ino != 0) + dirnode->inode->st.st_ctimensec = 0; + else + dirnode->inode->st.st_ctimensec = start_time.tv_nsec; #endif } if (specnode->flags & (F_UID | F_UNAME)) { -- 2.37.1 From c70510ba9edaf09b0ae5b8623826a83316fb0fc2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 11 Dec 2021 12:45:38 +0100 Subject: [PATCH 284/310] Revert "release/amd64/make-memstick.sh: Temporarily stop using the freshly build makefs" This reverts commit a1944e4166056f5a72f21af6b6c0ffc39a7b835c. --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 2a75e5bfb78d..39ecd8d9569b 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -55,7 +55,7 @@ MAKEFS_DEBUG_FLAGS=0x81400 # # Bind makefs to a single cpu as the output is core dependant # (XXX: doesn't seem to have the intended effect) -cpuset -l 0 makefs -T "${EPOCH_DATE-0}" \ +cpuset -l 0 dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ -d "${MAKEFS_DEBUG_FLAGS}" \ -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab -- 2.37.1 From 3d08c9fcbfa04e39b25e0a8c91c1cae57e0f586f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 11 Dec 2021 18:34:56 +0100 Subject: [PATCH 285/310] libexec/rc/rc.conf: Default to not warning about 'obsolete' per-jail configuration Obtained from: ElectroBSD --- libexec/rc/rc.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libexec/rc/rc.conf b/libexec/rc/rc.conf index b42811a4ddea..f4048b1dfcc7 100755 --- a/libexec/rc/rc.conf +++ b/libexec/rc/rc.conf @@ -731,7 +731,7 @@ iovctl_files="" # Config files for iovctl(8) ############################################################## jail_enable="NO" # Set to NO to disable starting of any jails jail_conf="/etc/jail.conf" # Configuration file for jail(8) -jail_confwarn="YES" # Prevent warning about obsolete per-jail configuration +jail_confwarn="NO" # Prevent warning about obsolete per-jail configuration jail_parallel_start="NO" # Start jails in the background jail_list="" # Space separated list of names of jails jail_reverse_stop="NO" # Stop jails in reverse order -- 2.37.1 From 4027e86917ac394a0bd42d5e2b3c5d704c1fa850 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 12 Dec 2021 15:29:18 +0100 Subject: [PATCH 286/310] libexec/rc/rc.d/jail: comment out useless warning that apparently is hard to disable with variables Obtained from: ElectroBSD --- libexec/rc/rc.d/jail | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/libexec/rc/rc.d/jail b/libexec/rc/rc.d/jail index a40080615be6..a96c6187a48b 100755 --- a/libexec/rc/rc.d/jail +++ b/libexec/rc/rc.d/jail @@ -151,11 +151,11 @@ parse_options() # : ${jail_confwarn:=YES} checkyesno jail_confwarn && _confwarn=1 - if [ -r "$jail_conf" -o -r "$_jconf" ]; then - if ! checkyesno jail_parallel_start; then - warn "$_conf is created and used for jail $_j." - fi - fi +# if [ -r "$jail_conf" -o -r "$_jconf" ]; then +# if ! checkyesno jail_parallel_start; then +# warn "$_conf is created and used for jail $_j." +# fi +# fi /usr/bin/install -m 0644 -o root -g wheel /dev/null $_conf || return 1 eval : \${jail_${_jv}_flags:=${jail_flags}} -- 2.37.1 From b2e947701fc69564bbef40fa66f329257096bca2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 18 Dec 2021 10:46:12 +0100 Subject: [PATCH 287/310] geli: Provide more details when the provider can't be attached ... due to mismatch between the expected and actual size. Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index 1de864257132..49f0b2bd5c3d 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -1060,7 +1060,10 @@ eli_attach(struct gctl_req *req) mediasize = g_get_mediasize(prov); if (md.md_provsize != (uint64_t)mediasize) { - gctl_error(r, "Provider size mismatch."); + gctl_error(r, "Provider size mismatch. Media size " + "is %llu but expected provider size is %llu.", + (unsigned long long)md.md_provsize, + (unsigned long long)mediasize); goto out; } -- 2.37.1 From b573cc7f4129bc3216f5953978be0a7bd29bfe69 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 16:27:31 +0100 Subject: [PATCH 288/310] sbin/geom/misc: Add g_metadata_search() ... which is needed to implement a "search" subcommand for geli. Obtained from: ElectroBSD --- sbin/geom/misc/subr.c | 75 +++++++++++++++++++++++++++++++++++++++++++ sbin/geom/misc/subr.h | 3 ++ 2 files changed, 78 insertions(+) diff --git a/sbin/geom/misc/subr.c b/sbin/geom/misc/subr.c index 3985ae56edc6..5141ce256c67 100644 --- a/sbin/geom/misc/subr.c +++ b/sbin/geom/misc/subr.c @@ -273,6 +273,81 @@ g_metadata_read(const char *name, unsigned char *md, size_t size, return (error); } +int +g_metadata_search(const char *name, unsigned char *md, size_t size, + const char *magic, off_t *metadata_offset) +{ + struct std_metadata stdmd; + unsigned char *sector; + ssize_t sectorsize; + off_t mediasize; + int error, fd; + off_t offset; + + *metadata_offset = 0; + sector = NULL; + error = 0; + + fd = g_open(name, 0); + if (fd == -1) + return (errno); + mediasize = g_mediasize(fd); + if (mediasize == -1) { + error = errno; + goto out; + } + sectorsize = g_sectorsize(fd); + if (sectorsize == -1) { + error = errno; + goto out; + } + assert(sectorsize >= (ssize_t)size); + sector = malloc(sectorsize); + if (sector == NULL) { + error = ENOMEM; + goto out; + } + + fprintf(stderr, "Searching for %s metadata on %s.\n", magic, name); + + offset = mediasize - sectorsize; + while (offset > 0) { + if (offset % 1073741824 == 0) { + /* Should be reworded */ + fprintf(stderr, + "Next search offset is %ju.\n", (uintmax_t)offset); + } + if (pread(fd, sector, sectorsize, offset) != + sectorsize) { + error = errno; + fprintf(stderr, "Failed to read at offset %ju.\n", + (uintmax_t)offset); + goto out; + } + if (magic != NULL) { + std_metadata_decode(sector, &stdmd); + if (strcmp(stdmd.md_magic, magic) != 0) { + offset -= sectorsize; + continue; + } else { + fprintf(stderr, + "Found %s meta data at offset %ju.\n", + magic, (uintmax_t)offset); + *metadata_offset = offset; + break; + } + } + offset -= sectorsize; + } + + bcopy(sector, md, size); +out: + if (sector != NULL) + free(sector); + g_close(fd); + return (error); +} + /* * Actually write the GEOM label to the provider * diff --git a/sbin/geom/misc/subr.h b/sbin/geom/misc/subr.h index 579c94c21491..9a05d47f2c6f 100644 --- a/sbin/geom/misc/subr.h +++ b/sbin/geom/misc/subr.h @@ -41,6 +41,9 @@ unsigned int g_get_sectorsize(const char *name); int g_metadata_read(const char *name, unsigned char *md, size_t size, const char *magic); +int +g_metadata_search(const char *name, unsigned char *md, size_t size, + const char *magic, off_t *metadata_offset); int g_metadata_store(const char *name, const unsigned char *md, size_t size); int g_metadata_clear(const char *name, const char *magic); -- 2.37.1 From def89e9f2091840ecdd676794405418c29e66362 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 16:35:00 +0100 Subject: [PATCH 289/310] geli: Implement a "search" subcommand ... which can be used to make a provider accessible again if, for example, the partition data is lost but the geli metadata on disk is still valid. Example usage: [fk@steffen ~]$ sudo geli search /dev/ada1 Searching for GEOM::ELI metadata on /dev/ada1. Found GEOM::ELI meta data at offset 4000785927680. Metadata found on /dev/ada1: magic: GEOM::ELI version: 7 flags: 0x0 ealgo: AES-XTS keylen: 128 provsize: 3985542778880 sectorsize: 512 keys: 0x01 iterations: 447024 Salt: 23[...]05 Master Key: 20[...]29 MD5 hash: 9fdffeb97ca6b34379512a9191cbfaeb Try making the data attachable with: gnop create -o 15243149312 -s 3985542778880 /dev/ada1 [fk@steffen ~]$ sudo gnop create -o 15243149312 -s 3985542778880 /dev/ada1 [fk@steffen ~]$ sudo geli attach /dev/ada1.nop Enter passphrase: [fk@steffen ~]$ sudo zpool import dpool [fk@steffen ~]$ sudo zpool status -v dpool pool: dpool state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://illumos.org/msg/ZFS-8000-8A scan: scrub repaired 0 in 1 days 09:00:47 with 66 errors on 2021-12-07 18:40:55 config: NAME STATE READ WRITE CKSUM dpool ONLINE 0 0 0 ada1.nop.eli ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: dpool/ggated/cloudia2:<0x1> dpool/ggated/cloudia2@2017-04-20_21:27:<0x1> Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 103 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index 49f0b2bd5c3d..0c6fe547a3f3 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -84,6 +84,7 @@ static void eli_resize(struct gctl_req *req); static void eli_version(struct gctl_req *req); static void eli_clear(struct gctl_req *req); static void eli_dump(struct gctl_req *req); +static void eli_search(struct gctl_req *req); static int eli_backup_create(struct gctl_req *req, const char *prov, const char *file); @@ -109,6 +110,7 @@ static int eli_backup_create(struct gctl_req *req, const char *prov, * version [prov ...] * clear [-v] prov ... * dump [-v] prov ... + * search [-v] prov ... */ struct g_command class_commands[] = { { "init", G_FLAG_VERBOSE, eli_main, @@ -283,6 +285,9 @@ struct g_command class_commands[] = { { "dump", G_FLAG_VERBOSE, eli_main, G_NULL_OPTS, "[-v] prov ..." }, + { "search", G_FLAG_VERBOSE, eli_main, G_NULL_OPTS, + "[-v] prov ..." + }, G_CMD_SENTINEL }; @@ -345,6 +350,8 @@ eli_main(struct gctl_req *req, unsigned int flags) eli_restore(req); else if (strcmp(name, "resize") == 0) eli_resize(req); + else if (strcmp(name, "search") == 0) + eli_search(req); else if (strcmp(name, "version") == 0) eli_version(req); else if (strcmp(name, "dump") == 0) @@ -656,6 +663,62 @@ eli_metadata_read(struct gctl_req *req, const char *prov, return (0); } +static int +eli_metadata_search(struct gctl_req *req, const char *prov, + struct g_eli_metadata *md, off_t *metadata_offset) +{ + unsigned char sector[sizeof(struct g_eli_metadata)]; + int error; + + if (g_get_sectorsize(prov) == 0) { + int fd; + + /* This is a file probably. */ + fd = open(prov, O_RDONLY); + if (fd == -1) { + gctl_error(req, "Cannot open %s: %s.", prov, + strerror(errno)); + return (-1); + } + if (read(fd, sector, sizeof(sector)) != sizeof(sector)) { + gctl_error(req, "Cannot read metadata from %s: %s.", + prov, strerror(errno)); + close(fd); + return (-1); + } + close(fd); + } else { + /* This is a GEOM provider. */ + error = g_metadata_search(prov, sector, sizeof(sector), + G_ELI_MAGIC, metadata_offset); + if (error != 0) { + gctl_error(req, "Cannot read metadata from %s: %s.", + prov, strerror(error)); + return (-1); + } + } + error = eli_metadata_decode(sector, md); + switch (error) { + case 0: + break; + case EOPNOTSUPP: + gctl_error(req, + "Provider's %s metadata version %u is too new.\n" + "geli: The highest supported version is %u.", + prov, (unsigned int)md->md_version, G_ELI_VERSION); + return (-1); + case EINVAL: + gctl_error(req, "Inconsistent provider's %s metadata.", prov); + return (-1); + default: + gctl_error(req, + "Unexpected error while decoding provider's %s metadata: %s.", + prov, strerror(error)); + return (-1); + } + return (0); +} + static int eli_metadata_store(struct gctl_req *req, const char *prov, struct g_eli_metadata *md) @@ -2012,3 +2075,43 @@ eli_dump(struct gctl_req *req) printf("\n"); } } + +static void +eli_suggest_gnop_command(const char *name, struct g_eli_metadata *md, + off_t metadata_offset) +{ + long long unsigned gnop_size = md->md_provsize; + long long unsigned gnop_offset = metadata_offset - md->md_provsize + + md->md_sectorsize; + + printf("\nTry making the data attachable with: " + "gnop create -o %llu -s %llu %s", + gnop_offset, gnop_size, name); +} + +static void +eli_search(struct gctl_req *req) +{ + struct g_eli_metadata md; + const char *name; + off_t metadata_offset; + int i, nargs; + + nargs = gctl_get_int(req, "nargs"); + if (nargs < 1) { + gctl_error(req, "Too few arguments."); + return; + } + + for (i = 0; i < nargs; i++) { + name = gctl_get_ascii(req, "arg%d", i); + if (eli_metadata_search(NULL, name, &md, &metadata_offset) == -1) { + gctl_error(req, "Not fully done."); + continue; + } + printf("Metadata found on %s:\n", name); + eli_metadata_dump(&md); + eli_suggest_gnop_command(name, &md, metadata_offset); + printf("\n"); + } +} -- 2.37.1 From a144f912d948bd8900809424fcefb61d0b816568 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 12 Nov 2021 03:56:15 +0100 Subject: [PATCH 290/310] geli(8): Document the "search" subcommand Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index dd3f059cdd13..c4e1b3e7c8c1 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -91,6 +91,9 @@ utility: .Op Fl bBdDgGrRtT .Ar prov ... .Nm +.Cm search +.Ar prov +.Nm .Cm setkey .Op Fl pPv .Op Fl i Ar iterations @@ -560,6 +563,13 @@ subcommand. .It Fl T Disable TRIM/UNMAP passthru. .El +.It Cm search +Search for metadata on the provider, starting at the end and +going backwards until valid metadata is found or the beginning +of the provider is reached. +This subcommand may be useful if, for example, the GPT partition +data got corrupted or deleted while the data on the previously +accessible partitions is still expected to be valid. .It Cm setkey Install a copy of the Master Key into the selected slot, encrypted with a new User Key. -- 2.37.1 From f1576e09cf4f7576ea99caed58cc1211771b7e68 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 25 Dec 2021 12:34:01 +0100 Subject: [PATCH 291/310] contrib/tzcode/stdtime: Optionally use non-standard ctime() format ... if the environment variable ISO_CTIME is set. Obtained from: ElectroBSD --- contrib/tzcode/stdtime/localtime.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/contrib/tzcode/stdtime/localtime.c b/contrib/tzcode/stdtime/localtime.c index 608842d5d432..1d5e1a3cd2b1 100644 --- a/contrib/tzcode/stdtime/localtime.c +++ b/contrib/tzcode/stdtime/localtime.c @@ -24,6 +24,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include "private.h" #include "un-namespace.h" @@ -1719,6 +1720,13 @@ struct tm * const tmp; char * ctime(const time_t *const timep) { + if (getenv("ISO_CTIME") != NULL) { + /* + * Use the non-standard time format that is + * considered easier to parse. + */ + return ctime_iso(timep); + } /* ** Section 4.12.3.2 of X3.159-1989 requires that ** The ctime function converts the calendar time pointed to by timer -- 2.37.1 From 09a0268089d5e998d786af8c9720bcf654f564fc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 25 Dec 2021 13:09:46 +0100 Subject: [PATCH 292/310] ctime.3: Document that ctime() checks the ISO_CTIME variable Obtained from: ElectroBSD --- contrib/tzcode/stdtime/ctime.3 | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/contrib/tzcode/stdtime/ctime.3 b/contrib/tzcode/stdtime/ctime.3 index 771027fa62ec..4198b91f1191 100644 --- a/contrib/tzcode/stdtime/ctime.3 +++ b/contrib/tzcode/stdtime/ctime.3 @@ -134,6 +134,15 @@ and returns a pointer to a 26-character string of the form: Thu Nov 24 18:22:48 1986\en\e0 .Ed .Pp +As a non-standard extension the +.Fn ctime +function checks the environment variable ISO_CTIME. +If it is set the fuction returns a +pointer to a 21-character string of the form: +.Bd -literal -offset indent +2021-11-01 22:52:37\en\e0 +.Ed +.Pp All the fields have constant width. .Pp The -- 2.37.1 From fa544a6e6494474b8f336596f9447e98f0593f98 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 28 Dec 2021 11:18:46 +0100 Subject: [PATCH 293/310] share/mk/src.opts.mk: Disable LLVM_TARGET_ALL and LLVM_CXXFILT (XXX: touches some more stuff) Obtained from: ElectroBSD --- share/mk/src.opts.mk | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 781fd695b045..466a8fba00ce 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -120,8 +120,6 @@ __DEFAULT_YES_OPTIONS = \ LLD_BOOTSTRAP \ LLD_IS_LD \ LLVM_COV \ - LLVM_CXXFILT \ - LLVM_TARGET_ALL \ LOADER_GELI \ LOADER_LUA \ LOADER_OFW \ @@ -184,6 +182,11 @@ __DEFAULT_NO_OPTIONS = \ HESIOD \ LIBSOFT \ LLVM_ASSERTIONS \ + LLVM_CXXFILT \ + LLVM_TARGET_ALL \ + LOADER_FIREWIRE \ + LOADER_FORCE_LE \ + LOADER_VERIEXEC_PASS_MANIFEST \ LLVM_BINUTILS \ LOADER_FIREWIRE \ LOADER_VERBOSE \ -- 2.37.1 From 68e46602bc75ae831f22ab25d521108baa2bec54 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 28 Dec 2021 11:51:52 +0100 Subject: [PATCH 294/310] Allow boostrapping on ElectroBSD again after 1d29feabb6e Obtained from: ElectroBSD --- usr.bin/clang/clang.prog.mk | 2 +- usr.bin/clang/lld/Makefile | 2 +- usr.bin/clang/llvm.prog.mk | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/usr.bin/clang/clang.prog.mk b/usr.bin/clang/clang.prog.mk index d0ed6b8587c5..5ba10f8ab7fc 100644 --- a/usr.bin/clang/clang.prog.mk +++ b/usr.bin/clang/clang.prog.mk @@ -17,7 +17,7 @@ LDADD+= ${OBJTOP}/lib/clang/lib${lib}/lib${lib}.a PACKAGE= clang -.if ${.MAKE.OS} == "FreeBSD" || !defined(BOOTSTRAPPING) +.if ${.MAKE.OS} == "ElectroBSD" || ${.MAKE.OS} == "FreeBSD" || !defined(BOOTSTRAPPING) LIBADD+= execinfo LIBADD+= ncursesw .endif diff --git a/usr.bin/clang/lld/Makefile b/usr.bin/clang/lld/Makefile index 9f114dbb81bf..9f52cf469982 100644 --- a/usr.bin/clang/lld/Makefile +++ b/usr.bin/clang/lld/Makefile @@ -99,7 +99,7 @@ DEPENDFILES+= ${TGHDRS:C/$/.d/} DPSRCS+= ${TGHDRS} CLEANFILES+= ${TGHDRS} ${TGHDRS:C/$/.d/} -.if ${.MAKE.OS} == "FreeBSD" || !defined(BOOTSTRAPPING) +.if ${.MAKE.OS} == "ElectroBSD" || ${.MAKE.OS} == "FreeBSD" || !defined(BOOTSTRAPPING) LIBADD+= execinfo LIBADD+= ncursesw .endif diff --git a/usr.bin/clang/llvm.prog.mk b/usr.bin/clang/llvm.prog.mk index 56698c4138d3..23df712e0014 100644 --- a/usr.bin/clang/llvm.prog.mk +++ b/usr.bin/clang/llvm.prog.mk @@ -22,7 +22,7 @@ LDADD+= ${OBJTOP}/lib/clang/lib${lib}/lib${lib}.a PACKAGE= clang -.if ${.MAKE.OS} == "FreeBSD" || !defined(BOOTSTRAPPING) +.if ${.MAKE.OS} == "ElectroBSD" || ${.MAKE.OS} == "FreeBSD" || !defined(BOOTSTRAPPING) LIBADD+= execinfo LIBADD+= ncursesw .endif -- 2.37.1 From 5205ad5277e4473ea0b43714a827c2311b43b3a7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Nov 2021 08:28:41 +0100 Subject: [PATCH 295/310] sys/geom/eli: Optionally turn on visibility for the last attemt ... ... to enter the passphrase. This is useful if the keyboard is prelling. Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 4534dfd7703a..9b2f59368fea 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -81,6 +81,10 @@ static u_int g_eli_visible_passphrase = GETS_NOECHO; SYSCTL_UINT(_kern_geom_eli, OID_AUTO, visible_passphrase, CTLFLAG_RWTUN, &g_eli_visible_passphrase, 0, "Visibility of passphrase prompt (0 = invisible, 1 = visible, 2 = asterisk)"); +static u_int g_eli_last_try_visible = 0; +SYSCTL_UINT(_kern_geom_eli, OID_AUTO, last_try_visible, CTLFLAG_RWTUN, + &g_eli_last_try_visible, 0, + "Make the entered passphrase visible if it's the last attempt"); u_int g_eli_overwrites = G_ELI_OVERWRITES; SYSCTL_UINT(_kern_geom_eli, OID_AUTO, overwrites, CTLFLAG_RWTUN, &g_eli_overwrites, 0, "Number of times on-disk keys should be overwritten when destroying them"); @@ -1299,8 +1303,15 @@ g_eli_taste(struct g_class *mp, struct g_provider *pp, int flags __unused) memcpy(passphrase, cached_passphrase, sizeof(passphrase)); } else { + showpass = g_eli_visible_passphrase; + if (g_eli_last_try_visible && + !showpass && i == tries) { + G_ELI_DEBUG(0, "Last try for %s. " + "Turning visibility on!", + pp->name); + showpass = 1; + } printf("Enter passphrase for %s: ", pp->name); - showpass = g_eli_visible_passphrase; if ((md.md_flags & G_ELI_FLAG_GELIDISPLAYPASS) != 0) showpass = GETS_ECHOPASS; cngets(passphrase, sizeof(passphrase), -- 2.37.1 From 58a5f7625e0c2a42341ae4db7f783f6c0d4b9ffe Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 26 Feb 2022 12:18:06 +0100 Subject: [PATCH 296/310] lib/geom/eli/geli.8: Document kern.geom.eli.last_try_visible Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index c4e1b3e7c8c1..2bf13221db28 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -882,6 +882,11 @@ passphrase can be logged and exposed via .Xr dmesg 8 . This variable should be set in .Pa /boot/loader.conf . +.It Va kern.geom.eli.last_try_visible : No 0 +If set to 1, the passphrase entered on boot will be visible +for the last try. This can be useful if the keyboard is prelling. +This variable should be set in +.Pa /boot/loader.conf . .It Va kern.geom.eli.threads : No 0 Specifies how many kernel threads should be used for doing software cryptography. -- 2.37.1 From 14ed2a1cd217e8f1d696b3232108b2a40899e384 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 20 Mar 2022 17:27:32 +0100 Subject: [PATCH 297/310] ELECTRO_BLOAT amd64: Mostly sync with GENERIC Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BEER | 5 +++++ sys/amd64/conf/ELECTRO_BLOAT | 41 ++++++++++++++++++------------------ 2 files changed, 25 insertions(+), 21 deletions(-) diff --git a/sys/amd64/conf/ELECTRO_BEER b/sys/amd64/conf/ELECTRO_BEER index 3fb26ab1675f..ba78781e085a 100644 --- a/sys/amd64/conf/ELECTRO_BEER +++ b/sys/amd64/conf/ELECTRO_BEER @@ -255,3 +255,8 @@ nodevice ses nodevice mrsas nodevice ixl nodevice ixlv + +# HID support +nooptions HID_DEBUG # enable debug msgs +nodevice hid # Generic HID support +nooptions IICHID_SAMPLING # Workaround missing GPIO INTR support diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index 4eaefc5d07bc..d9456755ec1f 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -32,6 +32,7 @@ options PREEMPTION # Enable kernel thread preemption options VIMAGE # Subsystem virtualization, e.g. VNET options INET # InterNETworking options INET6 # IPv6 communications protocols +options ROUTE_MPATH # Multipath routing support options TCP_OFFLOAD # TCP offload options TCP_BLACKBOX # Enhanced TCP event logging options TCP_HHOOK # hhook(9) framework for TCP @@ -51,6 +52,7 @@ options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework +options TMPFS # Efficient memory filesystem options GEOM_PART_GPT # GUID Partition Tables. # This is unlikely to be intentionally used be ElectroBSD # users but its mere presence in the kernel can result in @@ -62,6 +64,7 @@ options EFIRT # EFI Runtime Services support options COMPAT_FREEBSD32 # Compatible with i386 binaries options COMPAT_FREEBSD10 # Compatible with FreeBSD10 options COMPAT_FREEBSD11 # Compatible with FreeBSD11 +options COMPAT_FREEBSD12 # Compatible with FreeBSD12 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options STACK # stack(9) support @@ -109,13 +112,20 @@ device cpufreq # Bus support. device acpi -options ACPI_DMAR +device smbios +options IOMMU device pci options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support options COMPAT_LINUXKPI +# Enable support for the kernel PLL to use an external PPS signal, +# under supervision of [x]ntpd(8) +# More info in ntpd documentation: http://www.eecis.udel.edu/~ntp + +options PPS_SYNC + # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers @@ -130,12 +140,7 @@ device hptiop # Highpoint RocketRaid 3xxx series device mpt # LSI-Logic MPT-Fusion device mps # LSI-Logic MPT-Fusion 2 device mpr # LSI-Logic MPT-Fusion 3 -device trm # Tekram DC395U/UW/F DC315U adapters -device adv # Advansys SCSI adapters -device adw # Advansys wide SCSI adapters -device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. -device bt # Buslogic/Mylex MultiMaster SCSI adapters device isci # Intel C600 SAS controller # ATA/SCSI peripherals @@ -152,7 +157,6 @@ device ses # Enclosure Services (SES and SAF-TE) device amr # AMI MegaRAID device arcmsr # Areca SATA II RAID device ciss # Compaq Smart RAID 5* -device dpt # DPT Smartcache III, IV - See NOTES for options device iir # Intel Integrated RAID device ips # IBM (Adaptec) ServeRAID device mly # Mylex AcceleRAID/eXtremeRAID @@ -184,14 +188,15 @@ options VESA # Add support for VESA BIOS Extensions (VBE) device splash # Splash screen and screen saver support -# syscons is the default console driver, resembling an SCO console +# syscons is the legacy console driver, resembling an SCO console device sc options SC_PIXEL_MODE # add support for the raster text mode -# vt is the new video console driver +# vt is the default video console driver device vt device vt_vga device vt_efifb +device vt_vbefb device agp # support several AGP chipsets @@ -224,11 +229,8 @@ device iavf # Intel Adaptive Virtual Function device vmx # VMware VMXNET3 Ethernet # PCI Ethernet NICs. -device de # DEC/Intel DC21x4x (``Tulip'') device le # AMD Am7900 LANCE and Am79C9xx PCnet device ti # Alteon Networks Tigon I/II gigabit Ethernet -device txp # 3Com 3cR990 (``Typhoon'') -device vx # 3Com 3c590, 3c595 (``Vortex'') # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! @@ -244,32 +246,25 @@ device dc # DEC/Intel 21143 and various workalikes device et # Agere ET1310 10/100/Gigabit Ethernet device fxp # Intel EtherExpress PRO/100B (82557, 82558) device gem # Sun GEM/Sun ERI/Apple GMAC -device hme # Sun HME (Happy Meal Ethernet) device jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet device lge # Level 1 LXT1001 gigabit Ethernet device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet device nfe # nVidia nForce MCP on-board Ethernet device nge # NatSemi DP83820 gigabit Ethernet -device pcn # AMD Am79C97x PCI 10/100 (precedence over 'le') device re # RealTek 8139C+/8169/8169S/8110S device rl # RealTek 8129/8139 -device sf # Adaptec AIC-6915 (``Starfire'') device sge # Silicon Integrated Systems SiS190/191 device sis # Silicon Integrated Systems SiS 900/SiS 7016 device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet device ste # Sundance ST201 (D-Link DFE-550TX) device stge # Sundance/Tamarack TC9021 gigabit Ethernet -device tl # Texas Instruments ThunderLAN -device tx # SMC EtherPower II (83c170 ``EPIC'') device vge # VIA VT612x gigabit Ethernet device vr # VIA Rhine, Rhine II -device wb # Winbond W89C840F device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # Wireless NIC cards device wlan # 802.11 support options IEEE80211_DEBUG # enable debug msgs -options IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's options IEEE80211_SUPPORT_MESH # enable 802.11s draft support device wlan_wep # 802.11 WEP support device wlan_ccmp # 802.11 CCMP support @@ -279,7 +274,6 @@ device an # Aironet 4500/4800 802.11 wireless NICs. device ath # Atheros NICs device ath_pci # Atheros pci/cardbus glue device ath_hal # pci/cardbus chip support -options AH_SUPPORT_AR5416 # enable AR5416 tx/rx descriptors options AH_AR5416_INTERRUPT_MITIGATION # AR5416 interrupt mitigation options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later device ath_rate_sample # SampleRate tx rate control for ath @@ -289,8 +283,8 @@ device malo # Marvell Libertas wireless NICs. # Pseudo devices. device crypto # core crypto support +device aesni # AES-NI OpenCrypto module device loop # Network loopback -device random # Entropy device device padlock_rng # VIA Padlock RNG device rdrand_rng # Intel Bull Mountain RNG device ether # Ethernet support @@ -356,3 +350,8 @@ device netmap # netmap(4) support options EVDEV_SUPPORT # evdev support in legacy drivers device evdev # input event device support device uinput # install /dev/uinput cdev + +# HID support +options HID_DEBUG # enable debug msgs +device hid # Generic HID support +options IICHID_SAMPLING # Workaround missing GPIO INTR support -- 2.37.1 From 0c44f92333e014c3b21f2f17b4e109e46753435c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 20 Mar 2022 17:40:24 +0100 Subject: [PATCH 298/310] Makefile.inc1: Don't set CROSSBUILD_HOST on ElectroBSD Obtained from: ElectroBSD --- Makefile.inc1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.inc1 b/Makefile.inc1 index 16a82f93b116..5c1f8b2166ab 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -175,7 +175,7 @@ _t= ${TARGET_ARCH}/${TARGET} .endif .endfor -.if ${.MAKE.OS} != "FreeBSD" +.if ${.MAKE.OS} != "FreeBSD" && ${.MAKE.OS} != "ElectroBSD" CROSSBUILD_HOST=${.MAKE.OS} .if ${.MAKE.OS} != "Linux" && ${.MAKE.OS} != "Darwin" .warning "Unsupported crossbuild system: ${.MAKE.OS}. Build will probably fail!" -- 2.37.1 From 3ab8ddb755524325336607e8c1ceb7882723271c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 20 Mar 2022 17:46:44 +0100 Subject: [PATCH 299/310] tools/build/Makefile: Add ElectroBSD support Obtained from: ElectroBSD --- tools/build/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/build/Makefile b/tools/build/Makefile index 746f294aafba..3286b8aea93f 100644 --- a/tools/build/Makefile +++ b/tools/build/Makefile @@ -141,7 +141,7 @@ SYSINCS+= ${SRCTOP}/sys/sys/imgact_aout.h SYSINCS+= ${SRCTOP}/sys/sys/nlist_aout.h -.if ${.MAKE.OS} != "FreeBSD" +.if ${.MAKE.OS} != "FreeBSD" && ${.MAKE.OS} != "ElectroBSD" .PATH: ${.CURDIR}/cross-build # dbopen() behaves differently on Linux and FreeBSD so we ensure that we # bootstrap the FreeBSD db code. The cross-build headers #define dbopen() to @@ -276,7 +276,7 @@ _host_tools_to_symlink= basename bzip2 bunzip2 chmod chown cmp comm cp date dd \ _make_abs!= which "${MAKE}" _host_abs_tools_to_symlink= ${_make_abs}:make ${_make_abs}:bmake -.if ${.MAKE.OS} == "FreeBSD" +.if ${.MAKE.OS} == "FreeBSD" || ${.MAKE.OS} == "ElectroBSD" # When building on FreeBSD we always copy the host tools instead of linking # into WORLDTMP to avoid issues with incompatible libraries (see r364030). # Note: we could create links if we don't intend to update the current machine. -- 2.37.1 From aab19a2ef74754fb0f24bb785f08c03eecde8e49 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 20 Mar 2022 17:50:21 +0100 Subject: [PATCH 300/310] tools/build/mk/Makefile.boot: Add ElectroBSD support Obtained from: ElectroBSD --- tools/build/mk/Makefile.boot | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/build/mk/Makefile.boot b/tools/build/mk/Makefile.boot index 4e6f3508b30d..d4d764b04136 100644 --- a/tools/build/mk/Makefile.boot +++ b/tools/build/mk/Makefile.boot @@ -5,7 +5,7 @@ DPADD+= ${WORLDTMP}/legacy/usr/lib/libegacy.a LDADD+= -legacy LDFLAGS+= -L${WORLDTMP}/legacy/usr/lib -.if ${.MAKE.OS} != "FreeBSD" +.if ${.MAKE.OS} != "FreeBSD" && ${.MAKE.OS} != "ElectroBSD" # On MacOS using a non-mac ar will fail the build, similarly on Linux using # nm may not work as expected if the nm for the target architecture comes in # $PATH before a nm that supports the host architecture. -- 2.37.1 From 1da3723040eb1ff1526484c37803672a19f480b8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 22 Mar 2022 13:44:11 +0100 Subject: [PATCH 301/310] share/mk/src.opts.mk: Actually disable CXGBETOOL (XXX) Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 466a8fba00ce..2967933a4d5d 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -83,7 +83,6 @@ __DEFAULT_YES_OPTIONS = \ CRYPT \ CUSE \ CXX \ - CXGBETOOL \ DIALOG \ DICT \ DMAGENT \ @@ -240,6 +239,7 @@ __DEFAULT_NO_OPTIONS += \ BSDINSTALL \ CCD \ CTM \ + CXGBETOOL \ EXAMPLES \ FLOPPY \ FMTREE \ -- 2.37.1 From 8493630e0268015c985d4754898f4a22ceeb1d1d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 25 Mar 2022 12:10:24 +0100 Subject: [PATCH 302/310] zpool-upgrade.8: Remove superfluous 'the' Obtained from: ElectroBSD --- sys/contrib/openzfs/man/man8/zpool-upgrade.8 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/contrib/openzfs/man/man8/zpool-upgrade.8 b/sys/contrib/openzfs/man/man8/zpool-upgrade.8 index 1b13bad898bf..7721d05c85c5 100644 --- a/sys/contrib/openzfs/man/man8/zpool-upgrade.8 +++ b/sys/contrib/openzfs/man/man8/zpool-upgrade.8 @@ -64,7 +64,7 @@ property). .Cm upgrade .Fl v .Xc -Displays legacy ZFS versions supported by the this version of ZFS. +Displays legacy ZFS versions supported by this version of ZFS. See .Xr zpool-features 7 for a description of feature flags features supported by this version of ZFS. -- 2.37.1 From 0af089caf905b35e1b1598a4d723d3365292db97 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2022 21:32:29 +0200 Subject: [PATCH 303/310] ELECTRO_BLOAT amd64: Enable hacker movie colors for vt as well Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index d9456755ec1f..cbb6c84ce4e7 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -5,7 +5,10 @@ ident ELECTRO_BLOAT # One of ElectroBSD's most important features: # hacker-movie-compatible colors by default! +# First for sc: options SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK) +# Then for vt: +options TERMINAL_KERN_ATTR=(FG_GREEN|BG_BLACK) # Add HTTP accept filter support. The "performance gains" might # be dubious, but adding it results in nicer logs for applications -- 2.37.1 From 9ebe0d49c86bff37d3fa4e3e8a2451b36170cfb1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 1 Apr 2022 12:38:47 +0200 Subject: [PATCH 304/310] sys/vm/vm_pageout.c: Fix typo Obtained from: ElectroBSD --- sys/vm/vm_pageout.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/vm/vm_pageout.c b/sys/vm/vm_pageout.c index 74439d5884ef..4357dbd07e9a 100644 --- a/sys/vm/vm_pageout.c +++ b/sys/vm/vm_pageout.c @@ -1786,7 +1786,7 @@ vm_pageout_inactive(struct vm_domain *vmd, int shortage, int *addl_shortage) static int vm_pageout_oom_vote; /* - * The pagedaemon threads randlomly select one to perform the + * The pagedaemon threads randomly select one to perform the * OOM. Trying to kill processes before all pagedaemons * failed to reach free target is premature. */ -- 2.37.1 From e30cf9b7680d13fc0795c37e2487108dadd09f8e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 19 Apr 2022 16:28:47 +0200 Subject: [PATCH 305/310] stand: Do not force default values into the environment .. if the user does not specify teken.fg_color or teken.bg_color, allowing the kernel defaults to take effect. Patch by emaste@ taken from: https://reviews.freebsd.org/D34509 See also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=261311 Obtained from: ElectroBSD --- stand/efi/libefi/efi_console.c | 9 --------- stand/i386/libi386/vidconsole.c | 7 ------- 2 files changed, 16 deletions(-) diff --git a/stand/efi/libefi/efi_console.c b/stand/efi/libefi/efi_console.c index 1aeb94344c58..6b3528c206fd 100644 --- a/stand/efi/libefi/efi_console.c +++ b/stand/efi/libefi/efi_console.c @@ -1056,15 +1056,6 @@ cons_update_mode(bool use_gfx_mode) attr.ta_bgcolor = strtol(ptr, NULL, 10); teken_set_defattr(&gfx_state.tg_teken, &attr); - } else { - snprintf(env, sizeof(env), "%d", - attr.ta_fgcolor); - env_setenv("teken.fg_color", EV_VOLATILE, env, - efi_set_colors, env_nounset); - snprintf(env, sizeof(env), "%d", - attr.ta_bgcolor); - env_setenv("teken.bg_color", EV_VOLATILE, env, - efi_set_colors, env_nounset); } } } diff --git a/stand/i386/libi386/vidconsole.c b/stand/i386/libi386/vidconsole.c index 1ec6b4f5393a..e0656b86a65d 100644 --- a/stand/i386/libi386/vidconsole.c +++ b/stand/i386/libi386/vidconsole.c @@ -972,13 +972,6 @@ cons_update_mode(bool use_gfx_mode) attr.ta_bgcolor = strtol(ptr, NULL, 10); teken_set_defattr(&gfx_state.tg_teken, &attr); - } else { - snprintf(env, sizeof(env), "%d", attr.ta_fgcolor); - env_setenv("teken.fg_color", EV_VOLATILE, env, - vidc_set_colors, env_nounset); - snprintf(env, sizeof(env), "%d", attr.ta_bgcolor); - env_setenv("teken.bg_color", EV_VOLATILE, env, - vidc_set_colors, env_nounset); } /* Improve visibility */ -- 2.37.1 From 71a942802227bc9438b8ce6f89a395645a88902e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 17 May 2022 13:00:51 +0200 Subject: [PATCH 306/310] release/scripts/strip-freebsd.sh: Ditch iwlwififw files Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index fb0438300078..dc0fad2b63a5 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -75,6 +75,12 @@ get_bce_files() { "sys/dev/bce/if_bce.c" } +get_iwlwififw_files() { + if [ -d sys/contrib/dev/iwlwififw ]; then + find sys/contrib/dev/iwlwififw + fi +} + get_usb_firmware_files() { find sys/dev/usb/ -name "*fw*" } @@ -93,6 +99,7 @@ get_unfree_files() { get_snd_ds1_files get_snd_maestro3_files get_bce_files + get_iwlwififw_files get_usb_firmware_files } -- 2.37.1 From b6b2f3152507d223ac1cb44a426b58e431436fa2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 31 Mar 2022 13:36:11 +0200 Subject: [PATCH 307/310] tools/boot/install-boot.sh: Set the volume id to zero to get an reproducible esp file Obtained from: ElectroBSD --- tools/boot/install-boot.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/boot/install-boot.sh b/tools/boot/install-boot.sh index ce6e363a3067..51bcc7785f46 100755 --- a/tools/boot/install-boot.sh +++ b/tools/boot/install-boot.sh @@ -66,6 +66,7 @@ make_esp_file() { -o fat_type=${fatbits} \ -o sectors_per_cluster=1 \ -o volume_label=EFISYS \ + -o volume_id=0 \ -s ${sizekb}k \ "${file}" "${stagedir}" rm -rf "${stagedir}" -- 2.37.1 From 29a4609b2dbb1905cff3019db17417dd85d92372 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 29 May 2015 10:46:06 +0200 Subject: [PATCH 308/310] Import cloudiatr to 2022-07-17-329f9a9 Obtained from: ElectroBSD --- usr.sbin/Makefile | 1 + usr.sbin/cloudiatr/Makefile | 3 + usr.sbin/cloudiatr/cloudiatr | 1421 ++++++++++++++++++++++++++++++++++ 3 files changed, 1425 insertions(+) create mode 100644 usr.sbin/cloudiatr/Makefile create mode 100755 usr.sbin/cloudiatr/cloudiatr diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index 8fbbd4d56b4d..ff5b42d89812 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -7,6 +7,7 @@ SUBDIR= adduser \ arp \ binmiscctl \ camdd \ + cloudiatr \ cdcontrol \ chkgrp \ chown \ diff --git a/usr.sbin/cloudiatr/Makefile b/usr.sbin/cloudiatr/Makefile new file mode 100644 index 000000000000..34ada8582ecd --- /dev/null +++ b/usr.sbin/cloudiatr/Makefile @@ -0,0 +1,3 @@ +SCRIPTS= cloudiatr + +.include diff --git a/usr.sbin/cloudiatr/cloudiatr b/usr.sbin/cloudiatr/cloudiatr new file mode 100755 index 000000000000..ca577e39a13f --- /dev/null +++ b/usr.sbin/cloudiatr/cloudiatr @@ -0,0 +1,1421 @@ +#!/bin/sh + +########################################################################### +# cloudiatr +# +# Buzzword-compliant remote OS eviction tool. For details see: +# https://www.fabiankeil.de/gehacktes/cloudiatr/ +# +# Copyright (c) 2014-2022 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ALL YOUR +# DATA IS BELONG TO THE SOFTWARE AND MAY BE EATEN BY IT. IF THAT IS NOT +# ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. +########################################################################### + +# These subcommands skip the privilege check. +# +# The "cmd" subcommand may actually required privileges +# depending on the arguments but the check isn't strictly +# required and skipping it for all arguments is more convenient +# than enforcing it for all. +# +# The variable can't be set through cloudiatr_init_globals +# as it's needed earlier. +CLOUDIATR_UNPRIVILEGED_SUBCOMMANDS="cmd show-config version" + +# It's important that this function is called before any other +# function except cloudiatr_main(), otherwise fatal errors may +# not be caught. +cloudiatr_init() { + local mode="${1}" + + set -e + cloudiatr_init_globals + + cloudiatr_load_config_file "${CLOUDIATR_CONFIG_FILE}" + + cloudiatr_check_config +} + +cloudiatr_fyi() { + local message="${*}" + + echo "cloudiatr: $message" +} + +cloudiatr_wtf() { + local complaints="${*}" + if [ -z "${complaints}" ]; then + complaints="cloudiatr_wtf(): No complaints?" + fi + cloudiatr_fyi "${complaints}" 1>&2 + return 1 +} + +cloudiatr_check_config() { + local v_flag \ + mandatory_variable optional_variable value fail + + v_flag="${1}" + fail=0 + + for mandatory_variable in ${CLOUDIATR_MANDATORY_VARIABLES}; do + value="$(eval 'echo $'"${mandatory_variable}")" + if [ -z "${value}" ]; then + cloudiatr_wtf "Fatal error: ${mandatory_variable} is unset" + fail=1 + elif [ "${v_flag}" = "-v" ]; then + echo "${mandatory_variable}='${value}'" + fi + done + if [ "${v_flag}" = "-v" ]; then + for optional_variable in ${CLOUDIATR_OPTIONAL_VARIABLES}; do + value="$(eval 'echo $'"${optional_variable}")" + echo "${optional_variable}='${value}'" + done + fi + return $fail +} + +cloudiatr_show_config() { + cloudiatr_check_config -v +} + +cloudiatr_load_config_file() { + local config_file="${1}" + + if [ -f "${config_file}" ]; then + . "${config_file}" + return 0 + fi + cloudiatr_wtf "Config file ${config_file} does not exist. You can use '$0 -f path/to/file ...' to specify a different one" + return 1 +} + +cloudiatr_init_globals() { + + CLOUDIATR_VERSION="2022-07-17-329f9a9" + + CLOUDIATR_NEW_SYSTEM_DIR=/cloudiatr + # Only needs to be enough for a stripped-down bootfs + CLOUDIATR_BPOOL_PARTITION_SIZE=200M + # Has to be enough for the rest of the OS including the "permanent" + # /boot that is only used to (re)populate the bootfs on the bpool. + CLOUDIATR_RPOOL_PARTITION_SIZE=4G + CLOUDIATR_SWAP_PARTITION_SIZE=4G + + # Flags passed to zpool create when creating the boot and root pools. + # Creating the pools with version 28 makes recovery with outdated + # recovery systems more convenient. + CLOUDIATR_BPOOL_CREATE_FLAGS="-o version=28 -O compression=on -f" + CLOUDIATR_RPOOL_CREATE_FLAGS="-o version=28 -O compression=on -O checksum=sha256" + + # Set to true to use the existing partition layout. + # + # Only expected to work if the layout was created by a previous + # cloudiatr run. By default the partitions 2, 3 and 4 will be + # overwritten and partition 1 is expected to contain working bootcode. + CLOUDIATR_REUSE_GPART_SETUP=false + + # Usually changing the partition numbers usually is not necessary + # and changing them after the installation is likely to result in + # data loss. + CLOUDIATR_BOOTCODE_PARTITION=1 + CLOUDIATR_BPOOL_PARTITION=2 + CLOUDIATR_RPOOL_PARTITION=3 + CLOUDIATR_SWAP_PARTITION=4 + CLOUDIATR_DPOOL_PARTITION=5 + + CLOUDIATR_BPOOL_NAME="bpool" + CLOUDIATR_RPOOL_NAME="rpool" + + # Note that DEFAULT is a fallback documented in rc.conf(5). + # Not changing this variable to the actual network interface + # is likely to cause problems if there are more than one nics + # and you aren't using DHCP for all of them. + CLOUDIATR_NIC="DEFAULT" + + # Will be created + CLOUDIATR_RPOOL_KEY_NAME="${CLOUDIATR_RPOOL_NAME}.key" + CLOUDIATR_RPOOL_KEY="${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_KEY_NAME}" + + # If CLOUDIATR_DIST_IMAGE is set, an image containing the dist tarballs + # has to be put in place by the user before the eviction. If it's unset, + # CLOUDIATR_DIST_DIR has to be populated before cloudiatr is executed. + CLOUDIATR_DIST_IMAGE="" + CLOUDIATR_DIST_IMAGE_SHA256="" + CLOUDIATR_DIST_DIR="/usr/electrobsd-dist" + + # Whether or not the distribution tarballs should be copied to the + # newly installed system (for example to reuse them when setting + # up jails). + CLOUDIATR_SAVE_DIST_DIR="false" + + # Default to using all the detected ada(4) devices + CLOUDIATR_DISKS="$(cloudiatr_autodetect_disks)" + + # Changing these should only be necessary if there's more + # than one disk and you don't want to use a mirror. + CLOUDIATR_BPOOL_LAYOUT="default" + CLOUDIATR_RPOOL_LAYOUT="default" + + CLOUDIATR_GELI_KEY_LENGTH=256 + CLOUDIATR_GELI_EALGO=AES-XTS + # Use a passphrase for geli, this is currently interactive which isn't ideal + CLOUDIATR_GELI_USE_PASSPHRASE=false + + CLOUDIATR_SSHD_HOST_KEY_ALGORITHMS="rsa ecdsa ed25519" + + # Set to 'true' to ingore some safety-checks and increase the potential damage. + # Includes "geli kill -a" which is not limited to the disks specified above. + # Do not enable this unless the system that is being evicted doesn't contain + # any data you care about. + CLOUDIATR_MURDER_DEATH_KILL_REQUESTED=false + + # Set to 'true' to skip the image checksum check. + # "It may be insecure, but look how fast it is!" + CLOUDIATR_CHECKSUM_SMECKSUM=false + + # The config file is sourced and may overwrite any of the values above + # and most functions in this file (zogftw-style). + CLOUDIATR_CONFIG_FILE="${CLOUDIATR_CONFIG_FILE=/etc/cloudiatr.conf}" + + # A user that should be created and allowed to "su" on the new system. + CLOUDIATR_NEW_USER="cloudiatr" + + # This password is used for both root and CLOUDIATR_NEW_USER. + # + # Note that the created system will not accept root logins through + # ssh (FreeBSD default). CLOUDIATR_NEW_USER may use ssh, but has + # to use public key authentication. + CLOUDIATR_INITIAL_PASSWORD="${CLOUDIATR_NEW_USER}" + + # Local timezone. For details see tzsetup(8). + CLOUDIATR_TIME_ZONE="Europe/Berlin" + + # When set to true, cloudiatr will execute ntpdate at installtime. + # The server(s) being used depend on the install distfiles. + CLOUDIATR_USE_NTPDATE="false" + + # Any alignment should work, 1M is often recommended to prevent + # write-amplification which can result in performance degradation. + # It can also be advantageous for trimming SSDs. + CLOUDIATR_GPART_ALIGNMENT="1M" + + # Optional keyboard map for the virtual console. + # For details see kbdmap. + CLOUDIATR_KBDMAP="de.kbd" + + # Additional distributions to extract. Example: src, lib32 + CLOUDIATR_EXTRA_DISTRIBUTIONS="" + + # Value for rc.conf's rether_enable entry which controls + # whether or not MAC addresses are randomized (on ElectroBSD). + CLOUDIATR_RETHER_ENABLE="NO" + + # Set to true to not bother the user about with questions. + : "${CLOUDIATR_DONT_ASK_JUST_KISS=false}" + + # Set to true (default) to use the added swap partitions right + # after creating them. This allows installations on systems that + # have insufficient memory (512 MB, for example) and no previously + # configured swap devices. + # + # While this option is not expected to cause problems, if you + # are absolutely sure that enough memory is available you can + # disable the behaviour by setting the variable to "false". + CLOUDIATR_USE_SWAP_WHILE_INSTALLING="true" + + # Apply a workaround that is required to boot on + # Lenovo laptops like the T520 + CLOUDIATR_APPLY_LENOVO_WORKAROUND="false" + + # These subcommands are currently supported. + # The list is used by cloudiatr_show_usage() + CLOUDIATR_SUBCOMMANDS="clean-up evic recreate-bpool \ + soft-protect update update-base \ + update-bootcode update-kernel \ + version \ + ${CLOUDIATR_UNPRIVILEGED_SUBCOMMANDS}" + + # If these variables aren't set to some value, cloudiatr will abort. + # Sane values are a good idea but not mandatory. + CLOUDIATR_MANDATORY_VARIABLES="\ + CLOUDIATR_APPLY_LENOVO_WORKAROUND \ + CLOUDIATR_BPOOL_CREATE_FLAGS \ + CLOUDIATR_BPOOL_LAYOUT \ + CLOUDIATR_BPOOL_NAME \ + CLOUDIATR_BPOOL_PARTITION_SIZE \ + CLOUDIATR_CHECKSUM_SMECKSUM \ + CLOUDIATR_CONFIG_FILE \ + CLOUDIATR_DISKS \ + CLOUDIATR_DIST_DIR \ + CLOUDIATR_DONT_ASK_JUST_KISS \ + CLOUDIATR_GELI_KEY_LENGTH \ + CLOUDIATR_GELI_EALGO \ + CLOUDIATR_GELI_USE_PASSPHRASE \ + CLOUDIATR_GPART_ALIGNMENT \ + CLOUDIATR_HOSTNAME \ + CLOUDIATR_INITIAL_PASSWORD \ + CLOUDIATR_MURDER_DEATH_KILL_REQUESTED \ + CLOUDIATR_NEW_SYSTEM_DIR \ + CLOUDIATR_NEW_USER \ + CLOUDIATR_NIC \ + CLOUDIATR_USE_SWAP_WHILE_INSTALLING \ + CLOUDIATR_RETHER_ENABLE \ + CLOUDIATR_REUSE_GPART_SETUP \ + CLOUDIATR_RPOOL_CREATE_FLAGS \ + CLOUDIATR_RPOOL_KEY \ + CLOUDIATR_RPOOL_KEY_NAME \ + CLOUDIATR_RPOOL_LAYOUT \ + CLOUDIATR_RPOOL_NAME \ + CLOUDIATR_RPOOL_PARTITION_SIZE \ + CLOUDIATR_SAVE_DIST_DIR \ + CLOUDIATR_SWAP_PARTITION_SIZE \ + CLOUDIATR_TIME_ZONE \ + CLOUDIATR_USE_NTPDATE \ + CLOUDIATR_VERSION \ + " + + # These variables are allowed to be unset + CLOUDIATR_OPTIONAL_VARIABLES="\ + CLOUDIATR_DEFAULTROUTER \ + CLOUDIATR_DIST_IMAGE \ + CLOUDIATR_DIST_IMAGE_SHA256 \ + CLOUDIATR_EXTRA_DISTRIBUTIONS \ + CLOUDIATR_IP_ADDRESS \ + CLOUDIATR_KBDMAP \ + CLOUDIATR_NETMASK \ + " +} + +# Apply the workaround described at: +# https://lists.freebsd.org/pipermail/freebsd-i386/2013-March/010437.html +cloudiatr_apply_lenovo_workaround() { + local disk="${1}" \ + slice_table_original slice_table_new \ + partion_spec cylinders sectors_per_track cylinders_new heads + + if [ -z "${disk}" ]; then + cloudiatr_wtf "cloudiatr_apply_lenovo_workaround: No disk provided" + return 1 + fi + + slice_table_original=$(mktemp -t cloudiatr_slice_table_original) || return 1 + slice_table_new=$(mktemp -t cloudiatr_slice_table_new) || return 1 + + fdisk -p "${disk}" > "${slice_table_original}" || return 1 + + cylinders="$(grep '^g c' "${slice_table_original}" | cut -w -f 2 | cut -d c -f 2)" || return 1 + heads="$(grep '^g c' "${slice_table_original}" | cut -w -f 3 | cut -d h -f 2)" || return 1 + sectors_per_track="$(grep '^g c' "${slice_table_original}" | cut -w -f 4 | cut -d s -f 2)" || return 1 + + cylinders_new="$(expr "${cylinders}" \* "${sectors_per_track}")" || return 1 + + partition_spec="$(tail -n 1 ${slice_table_original})" || return 1 + + echo "# ${disk}" > "${slice_table_new}" || return 1 + echo "g c${cylinders_new} h${heads} s1" >> "${slice_table_new}" || return 1 + echo "${partition_spec}" | sed -e 's@^p 1 0xee@p 1 0x00@' >> "${slice_table_new}" || return 1 + echo "a 1" >> "${slice_table_new}" || return 1 + echo "${partition_spec}" | sed -e 's@^p 1@p 2@' >> "${slice_table_new}" || return 1 + + cloudiatr_fyi "Applying Lenovo workaround:" + diff -u "${slice_table_original}" "${slice_table_new}" || true + fdisk -f "${slice_table_new}" "${disk}" || return 1 + rm "${slice_table_original}" "${slice_table_new}" || return 1 +} + +cloudiatr_install_bootcode_on_disk() { + local boot_dir disk + + boot_dir="${1}" #/boot + disk="${2}" + + cloudiatr_fyi "Installing boot code from ${boot_dir} on ${disk} ..." + gpart bootcode -b "${boot_dir}/pmbr" -p "${boot_dir}/gptzfsboot" \ + -i "${CLOUDIATR_BOOTCODE_PARTITION}" "${disk}" || return 1 +} + +cloudiatr_install_bootcode() { + local boot_dir \ + disk + + boot_dir="${1}" + + for disk in ${CLOUDIATR_DISKS}; do + cloudiatr_install_bootcode_on_disk "${boot_dir}" "${disk}" || return 1 + done +} + +cloudiatr_gpart_disk() { + local disk="${1}" \ + disk_name + + disk_name="${disk##*/}" + + gpart create -s gpt "${disk}" || return 1 + + gpart add -s 512 -t freebsd-boot \ + -i "${CLOUDIATR_BOOTCODE_PARTITION}" "${disk}" || return 1 + gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot \ + -i "${CLOUDIATR_BOOTCODE_PARTITION}" "${disk}" || return 1 + + gpart add -s "${CLOUDIATR_BPOOL_PARTITION_SIZE}" -a "${CLOUDIATR_GPART_ALIGNMENT}" \ + -l "${CLOUDIATR_BPOOL_NAME}-${disk_name}" -t freebsd-zfs \ + -i "${CLOUDIATR_BPOOL_PARTITION}" "${disk}" || return 1 + gpart add -s "${CLOUDIATR_RPOOL_PARTITION_SIZE}" -a "${CLOUDIATR_GPART_ALIGNMENT}" \ + -l "${CLOUDIATR_RPOOL_NAME}-${disk_name}" -t freebsd-zfs \ + -i "${CLOUDIATR_RPOOL_PARTITION}" "${disk}" || return 1 + gpart add -s "${CLOUDIATR_SWAP_PARTITION_SIZE}" -a "${CLOUDIATR_GPART_ALIGNMENT}" \ + -l "swap-${disk_name}" -t freebsd-swap \ + -i "${CLOUDIATR_SWAP_PARTITION}" "${disk}" || return 1 + + # Reserve what's left for the data pool + gpart add -l "dpool-${disk_name}" -a "${CLOUDIATR_GPART_ALIGNMENT}" \ + -t freebsd-zfs -i "${CLOUDIATR_DPOOL_PARTITION}" "${disk}" || return 1 + + if "${CLOUDIATR_APPLY_LENOVO_WORKAROUND}"; then + cloudiatr_apply_lenovo_workaround "${disk}" || return 1 + fi +} + +cloudiatr_gpart_setup() { + local disks d + + disks="${*}" + + cloudiatr_fyi "Cleaning partition tables (if there are any) ..." + for d in $disks; do + gpart destroy -F "${d}" 2>/dev/null || true + done + + cloudiatr_fyi "Partitioning disks ..." + for d in $disks; do + cloudiatr_gpart_disk "${d}" || return 1 + done +} + +# Use the swap partitions on the given disks while cloudiatr is running. +# This allows to install on a system with 512MB RAM or less and no swap space. +cloudiatr_enable_swap() { + local disks d + + disks="${*}" + + cloudiatr_fyi "Using created swap space while installing ..." + for d in $disks; do + geli onetime -d "${d}p${CLOUDIATR_SWAP_PARTITION}" || return 1 + swapon "${d}p${CLOUDIATR_SWAP_PARTITION}.eli" || return 1 + done +} + +cloudiatr_disable_swap() { + local disks d + + disks="${*}" + + cloudiatr_fyi "Trying to disable previously added swap space ..." + for d in $disks; do + swapoff "${d}p${CLOUDIATR_SWAP_PARTITION}.eli" || return 1 + done +} + +cloudiatr_get_geoms() { + local postfix="${1}" \ + d geoms + + for d in ${CLOUDIATR_DISKS}; do + geom_partition="${d}${postfix}" + geoms="${geoms} ${geom_partition}" + done + echo "${geoms## }" +} + +cloudiatr_get_gpart_labels() { + local postfix="${1}" + + for d in ${CLOUDIATR_DISKS}; do + d="${d##/dev/}" + gpart show -l -p "${d}" 2>/dev/null | awk '$3 == "'"${d}${postfix}"'" {printf "%s ", $4}' + done + echo +} + +# Depends on geli being already setup +cloudiatr_create_rpool() { + local \ + rpool_elis pool_layout + + rpool_elis="$(cloudiatr_get_geoms "p${CLOUDIATR_RPOOL_PARTITION}.eli")" + pool_layout="${CLOUDIATR_RPOOL_LAYOUT}" + + if [ "${pool_layout}" = "default" ]; then + pool_layout="$(cloudiatr_get_default_pool_layout)" + fi + + cloudiatr_fyi "Creating root pool '${CLOUDIATR_RPOOL_NAME}' on ${rpool_elis}. Pool layout: ${pool_layout}" + zpool create ${CLOUDIATR_RPOOL_CREATE_FLAGS} \ + "${CLOUDIATR_RPOOL_NAME}" ${pool_layout##single-disk} $rpool_elis + + zfs set mountpoint="${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}" "${CLOUDIATR_RPOOL_NAME}" + + zfs create "${CLOUDIATR_RPOOL_NAME}/boot" + # We currently use no dedicated dataset for /etc as the kernel expects parts of it + # to be available once the rootfs has been mounted. Having two /etc's can be a bit + # of a hassle on updates and thus doesn't seem like a good default. + #zfs create -o setuid=off "${CLOUDIATR_RPOOL_NAME}/etc" + zfs create "${CLOUDIATR_RPOOL_NAME}/home" + zfs create "${CLOUDIATR_RPOOL_NAME}/home/${CLOUDIATR_NEW_USER}" + zfs create -o exec=on -o setuid=off "${CLOUDIATR_RPOOL_NAME}/tmp" + zfs create "${CLOUDIATR_RPOOL_NAME}/usr" + zfs create -o compression=gzip -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/src" + zfs create "${CLOUDIATR_RPOOL_NAME}/var" + zfs create "${CLOUDIATR_RPOOL_NAME}/usr/local" + zfs create -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/local/etc" + zfs create -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/local/src" + zfs create -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/ports" + zfs create -o compression=off -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/ports/distfiles" + zfs create -o compression=off -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/ports/packages" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/cache" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/cache/pkg" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/crash" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/db" + zfs create -o exec=on -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/db/pkg" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/empty" + zfs create -o compression=gzip -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/log" + zfs create -o compression=gzip -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/mail" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/run" + zfs create -o exec=on -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/tmp" + + chmod 0750 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/crash" + chgrp mail "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/mail" + chmod 0775 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/mail" + chmod 0555 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/empty" + chflags schg,nouarch "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/empty" + chmod 1777 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/tmp" + chmod 1777 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/tmp" + + zfs set readonly=on "${CLOUDIATR_RPOOL_NAME}/var/empty" +} + +cloudiatr_kernel_module_is_loaded() { + local module="${1}" + # Can't use 'kldstat -m foo' as it requires a module file on disk. WTF? + kldstat | grep -q "${module}" +} + +cloudiatr_mount_dist_image() { + # intentionally leaks non-local variable md + + if [ ! -f "${CLOUDIATR_DIST_IMAGE}" ]; then + cloudiatr_wtf "File ${CLOUDIATR_DIST_IMAGE} does not exist" + return 1 + fi + + if "${CLOUDIATR_CHECKSUM_SMECKSUM}"; then + cloudiatr_fyi "Checksum smecksum" + else + cloudiatr_fyi "Checking checksum for image file ${CLOUDIATR_DIST_IMAGE} ..." + sha256 -c "${CLOUDIATR_DIST_IMAGE_SHA256}" "${CLOUDIATR_DIST_IMAGE}" + fi + md=$(mdconfig -f "${CLOUDIATR_DIST_IMAGE}") + if [ -z "${md}" ]; then + return 1 + fi + + for potential_partition in "/dev/${md}a" "/dev/${md}p2"; do + if [ -c "${potential_partition}" ]; then + cloudiatr_fyi "Trying to mount ${potential_partition} ..." + mount -o ro "${potential_partition}" /mnt/ || return 1 + cloudiatr_fyi "Mounting ${potential_partition} worked..." + fi + done +} + +cloudiatr_extract_distribution() { + local chroot_dir \ + md extra_distribution + + chroot_dir="${1}" + + if [ -n "${CLOUDIATR_DIST_IMAGE}" ]; then + cloudiatr_mount_dist_image + else + cloudiatr_fyi "No CLOUDIATR_DIST_IMAGE specified." + cloudiatr_fyi "Using CLOUDIATR_DIST_DIR=${CLOUDIATR_DIST_DIR}!" + fi + + cloudiatr_fyi "Extracting base in ${chroot_dir} ..." + # Exclude /var/empty as it's read-only + (cd "${chroot_dir}" && tar xpf "${CLOUDIATR_DIST_DIR}/base.txz" \ + --safe-writes --exclude ./var/empty/ --exclude ./net/ --exclude ./base/) + + cloudiatr_fyi "Creating directories that were missing in the base tarball" + chroot "${chroot_dir}" mtree -f /etc/mtree/BSD.root.dist -d -e -U + chroot "${chroot_dir}" mtree -f /etc/mtree/BSD.var.dist -d -e -U -p var + + for extra_distribution in ${CLOUDIATR_EXTRA_DISTRIBUTIONS}; do + cloudiatr_fyi "Extracting extra distribution '${extra_distribution}'" + (cd "${chroot_dir}" && tar xpf "${CLOUDIATR_DIST_DIR}/${extra_distribution}.txz") + done + + cloudiatr_fyi "Extracting kernel (without symbols) in ${chroot_dir} ..." + (cd "${chroot_dir}" && tar xpf "${CLOUDIATR_DIST_DIR}/kernel.txz" --exclude "*.symbols") + + if [ -n "${CLOUDIATR_DIST_IMAGE}" ]; then + umount /mnt + mdconfig -d -u ${md##md} + fi +} + +cloudiatr_setup_new_user() { + local ssh_dir="${chroot_dir}/home/${CLOUDIATR_NEW_USER}/.ssh" + + cloudiatr_fyi "Creating user '${CLOUDIATR_NEW_USER}'" + + echo "${CLOUDIATR_INITIAL_PASSWORD}" | chroot "${chroot_dir}" \ + pw useradd "${CLOUDIATR_NEW_USER}" -G wheel,operator -h 0 + + # Make sure the user can login through ssh, using the + # authorized_keys file from the installation media. + # + # An extra distribution file may already have created + # the .ssh directory, so don't fail if it already exits. + mkdir -p "${ssh_dir}" + if [ -f "${HOME}/.ssh/authorized_keys" ]; then + cp -v "${HOME}/.ssh/authorized_keys" "${ssh_dir}" + fi + chroot "${chroot_dir}" chown -R "${CLOUDIATR_NEW_USER}" "/home/${CLOUDIATR_NEW_USER}" + chroot "${chroot_dir}" chmod -R go-rwx "/home/${CLOUDIATR_NEW_USER}" +} + +cloudiatr_create_geli_key() { + local keyfile="${1}" + + ( + umask 077 + dd bs=64 count=1 if=/dev/random of="${keyfile}" 2>/dev/null + ) +} + +cloudiatr_setup_geli() { + local disks \ + d geli_passphrase_flag + + disks="${*}" + geli_no_passphrase_flag="-P" + + cloudiatr_create_geli_key "${CLOUDIATR_RPOOL_KEY}" + + mkdir "${CLOUDIATR_NEW_SYSTEM_DIR}/geli-backups" + + for d in $disks; do + if $CLOUDIATR_GELI_USE_PASSPHRASE; then + geli_no_passphrase_flag="" + fi + cloudiatr_fyi "Initialising geli on ${d}p${CLOUDIATR_RPOOL_PARTITION} ..." + geli init -b $geli_no_passphrase_flag \ + -B "${CLOUDIATR_NEW_SYSTEM_DIR}/geli-backups/${d##/dev/}p${CLOUDIATR_RPOOL_PARTITION}.eli" \ + -K "${CLOUDIATR_RPOOL_KEY}" -l "${CLOUDIATR_GELI_KEY_LENGTH}" \ + -e "${CLOUDIATR_GELI_EALGO}" -s 4096 -V 7 "${d}p${CLOUDIATR_RPOOL_PARTITION}" > /dev/null + done + + cloudiatr_attach_geli_geoms "${CLOUDIATR_RPOOL_KEY}" "p${CLOUDIATR_RPOOL_PARTITION}" +} + +cloudiatr_attach_geli_geoms() { + local keyfile partition_id \ + disk geli_passphrase_flag + + keyfile="${1}" + partition_id="${2}" + geli_no_passphrase_flag="-p" + + for disk in ${CLOUDIATR_DISKS}; do + if $CLOUDIATR_GELI_USE_PASSPHRASE; then + geli_no_passphrase_flag="" + fi + cloudiatr_fyi "geli-attaching ${disk}${partition_id}" + geli attach $geli_no_passphrase_flag -k "$keyfile" "${disk}${partition_id}" + done +} + +cloudiatr_detach_geli_geoms() { + local partition_id + + partition_id="${1}" + + for disk in ${CLOUDIATR_DISKS}; do + cloudiatr_fyi "geli-detaching ${disk}${partition_id}" + geli detach "${disk}${partition_id}.eli" + done +} + +cloudiatr_get_bpool_geoms() { + cloudiatr_get_geoms "p${CLOUDIATR_BPOOL_PARTITION}" +} + +cloudiatr_get_disk_names() { + local disk + for disk in ${CLOUDIATR_DISKS}; do + echo "${disk##*/}" + done +} + +cloudiatr_get_number_of_disks() { + local \ + disk number_of_disks + + number_of_disks=0 + for disk in ${CLOUDIATR_DISKS}; do + number_of_disks=$((number_of_disks+1)) + done + echo "${number_of_disks}" +} + +cloudiatr_get_default_pool_layout() { + if [ "$(cloudiatr_get_number_of_disks)" = 1 ]; then + echo "single-disk" + else + echo "mirror" + fi +} + +cloudiatr_autodetect_disks() { + local \ + disk + + for disk in $(sysctl -n kern.disks); do + # Only use ada(4) devices. We obviously can't use cd(4) + # devices and using da(4) devices would require us to + # skip the one we (probably) booted from. + if [ "${disk##ada}" != "${disk}" ]; then + echo "/dev/${disk}" + fi + done +} + +cloudiatr_create_bpool() { + local \ + bpool_geoms pool_layout + + bpool_geoms="$(cloudiatr_get_bpool_geoms)" + pool_layout="${CLOUDIATR_BPOOL_LAYOUT}" + + if [ "${pool_layout}" = "default" ]; then + pool_layout="$(cloudiatr_get_default_pool_layout)" + fi + + cloudiatr_fyi "Creating boot pool '${CLOUDIATR_BPOOL_NAME}' on ${bpool_geoms}. Pool layout: ${pool_layout}" + zpool create ${CLOUDIATR_BPOOL_CREATE_FLAGS} \ + "${CLOUDIATR_BPOOL_NAME}" ${pool_layout##single-disk} \ + $bpool_geoms + + # This currently can't be set at create-time + zpool set "bootfs=${CLOUDIATR_BPOOL_NAME}" "${CLOUDIATR_BPOOL_NAME}" + + # Would be nice, but for the bootfs to work, + # its ./boot directory can't be a zfs fs. + # + # XXX: can we work around this by setting bootfs + # on bpool/boot and use a symlink from bootf/boot/boot + # to bootf/boot? + # zfs create "${CLOUDIATR_BPOOL_NAME}/boot" +} + +cloudiatr_recreate_bpool() { + local boot_dir="${1}" + + cloudiatr_create_bpool || return 1 + cloudiatr_populate_bpool "${boot_dir}" || return 1 +} + +cloudiatr_setup_tmpfs() { + mkdir -p "${CLOUDIATR_NEW_SYSTEM_DIR}" + mount -t tmpfs tmpfs "${CLOUDIATR_NEW_SYSTEM_DIR}" +} + +cloudiatr_generate_rc_conf() { + local \ + netmask + + cat < "${config_file}" +} + +cloudiatr_create_config_files() { + local chroot_dir="${1}" + + cloudiatr_generate_file loader_conf "${chroot_dir}/boot/loader.conf" + cloudiatr_generate_file sysctl_conf "${chroot_dir}/etc/sysctl.conf" + cloudiatr_generate_file rc_conf "${chroot_dir}/etc/rc.conf" + cloudiatr_generate_file fstab "${chroot_dir}/etc/fstab" + cloudiatr_generate_file resolv_conf "${chroot_dir}/etc/resolv.conf" || true +} + +cloudiatr_get_required_kernel_content() { + kldstat | awk '/k/ {print $5}' +} + +# XXX: May creates output with duplicated slashes. Ugly but harmless. +# XXX: Why do we ignore errors here? +cloudiatr_populate_bpool() { + local boot_dir \ + boot_file new_kernel_dir sub_dir bpool_mountpoint new_file + + boot_dir="${1}" + if [ "${boot_dir}" = "/" ]; then + # Prevent duplicated leading slash in log messages + boot_dir="" + fi + bpool_mountpoint="${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_BPOOL_NAME}" + + cloudiatr_fyi "Populating boot pool '${CLOUDIATR_BPOOL_NAME}' ..." + zfs set mountpoint="${bpool_mountpoint}" "${CLOUDIATR_BPOOL_NAME}" || return 1 + + # Only copy what we really need before we can import the encrypted rpool + for sub_dir in defaults dtb firmware fonts images kernel kernel.old lua modules zfs; do + mkdir -p "${bpool_mountpoint}/boot/${sub_dir}" + done + + for boot_file in $(cloudiatr_get_required_kernel_content); do + # XXX: Create missing directories here + cp -v "${boot_dir}/boot/kernel/${boot_file}" "${bpool_mountpoint}/boot/kernel/" || true + if [ -f "${boot_dir}/boot/kernel.old/${boot_file}" ]; then + cp -v "${boot_dir}/boot/kernel.old/${boot_file}" "${bpool_mountpoint}/boot/kernel.old/" || true + fi + done + + for boot_file in $(find -s "${boot_dir}/boot/" \ + -not -path "${boot_dir}/boot/kernel*" -a \ + -not -path "${boot_dir}/boot/boot*" -a \ + -not -path "${boot_dir}/boot/cdboot" -a \ + -not -path "${boot_dir}/boot/*mbr" -a \ + -type f); do + new_file="${bpool_mountpoint}/boot/${boot_file##*/boot/}" + cp -v "${boot_file}" "${new_file}" || true + done + + zfs set "mountpoint=/${CLOUDIATR_BPOOL_NAME}" "${CLOUDIATR_BPOOL_NAME}" +} + +cloudiatr_clean_up() { + cloudiatr_fyi "Exporting boot pool '${CLOUDIATR_BPOOL_NAME}' ..." + zpool export "${CLOUDIATR_BPOOL_NAME}" + cloudiatr_fyi "Exporting root pool '${CLOUDIATR_RPOOL_NAME}' ..." + zpool export "${CLOUDIATR_RPOOL_NAME}" + cloudiatr_detach_geli_geoms "p${CLOUDIATR_RPOOL_PARTITION}" + umount "${CLOUDIATR_NEW_SYSTEM_DIR}" + rmdir "${CLOUDIATR_NEW_SYSTEM_DIR}" + if "${CLOUDIATR_USE_SWAP_WHILE_INSTALLING}"; then + if ! cloudiatr_disable_swap ${CLOUDIATR_DISKS}; then + cloudiatr_fyi "Failed to remove all the added swap space." + cloudiatr_fyi "If the system is low on memory the problem can be safely ignored." + fi + fi +} + +cloudiatr_generate_ssh_hostkeys() { + local chroot_dir \ + real_hostname + + chroot_dir="${1}" + real_hostname="$(hostname)" + + hostname "${CLOUDIATR_HOSTNAME}" + cloudiatr_fyi "Generating ssh host keys for ${CLOUDIATR_HOSTNAME} ..." + for key_alg in ${CLOUDIATR_SSHD_HOST_KEY_ALGORITHMS}; do + key_file="${chroot_dir}/etc/ssh/ssh_host_${key_alg}_key" + if ! ssh-keygen -q -t "${key_alg}" -f "${key_file}" -N ""; then + if [ "${key_alg}" = "ed25519" ]; then + # ed25519 isn't supported on FreeBSD 10.0 and earlier, + # thus we allow this to fail + continue + fi + return 1 + fi + ssh-keygen -l -v -f "${key_file}.pub" + done + hostname "${real_hostname}" +} + +cloudiatr_collect_evidence() { + local \ + evidence_dataset evidence disk_name + + evidence_dataset="${CLOUDIATR_RPOOL_NAME}/cloudiatr-evidence" + + cloudiatr_fyi "Collecting 'evidence' in /${evidence_dataset} ..." + zfs create "${evidence_dataset}" + for evidence in "${CLOUDIATR_CONFIG_FILE}" "${0}" \ + "${CLOUDIATR_NEW_SYSTEM_DIR}/geli-backups/"* \ + "${CLOUDIATR_RPOOL_KEY}"; do + cp -p "${evidence}" "${CLOUDIATR_NEW_SYSTEM_DIR}/${evidence_dataset}" + done + + for disk_name in $(cloudiatr_get_disk_names); do + evidence="${CLOUDIATR_NEW_SYSTEM_DIR}/${evidence_dataset}/${disk_name}.gpart" + gpart backup "${disk_name}" > "${evidence}" + done + + if "${CLOUDIATR_SAVE_DIST_DIR}"; then + cp -r "${CLOUDIATR_DIST_DIR%%/}" "${CLOUDIATR_NEW_SYSTEM_DIR}/${evidence_dataset}/" + fi + + find "${CLOUDIATR_NEW_SYSTEM_DIR}/${evidence_dataset}" -type f | sort +} + +cloudiatr_generate_sshd_config_extension() { + cat <> ${chroot_dir}/etc/ssh/sshd_config + + if "${CLOUDIATR_USE_NTPDATE}"; then + chroot ${chroot_dir} service ntpdate onestart || true + fi + + umount ${chroot_dir}/dev/ + + cloudiatr_setup_new_user + + if [ -f /boot/zfs/zpool.cache ]; then + # Copying the zpool.cache is no longer necessary on ElectroBSD + # and recent FreeBSD versions, but doesn't hurt. + cp /boot/zfs/zpool.cache "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/boot/zfs/zpool.cache" + fi + + cloudiatr_populate_bpool "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}" + + cloudiatr_collect_evidence + + cloudiatr_fyi "Setting final mountpoints on root pool '${CLOUDIATR_RPOOL_NAME}' ..." + zfs umount "${CLOUDIATR_RPOOL_NAME}" + zfs set mountpoint=legacy "${CLOUDIATR_RPOOL_NAME}" + for fs in boot home tmp usr var; do + zfs set "mountpoint=/${fs}" "${CLOUDIATR_RPOOL_NAME}/${fs}" + done + + cloudiatr_clean_up +} + +# The murder-death-kill feature was added for testing. After the introduction of +# the boring "clean-up" subcommand (which doesn't involve killing) it could be +# considered obsolete, but keeping it around makes cloudiatr more awesome. +cloudiatr_murder_death_kill() { + if "${CLOUDIATR_MURDER_DEATH_KILL_REQUESTED}"; then + cloudiatr_fyi "You really asked for it. Murder death kill in progress ..." + zpool export "${CLOUDIATR_BPOOL_NAME}" || true + zpool export "${CLOUDIATR_RPOOL_NAME}" || true + umount "${CLOUDIATR_NEW_SYSTEM_DIR}" || true + geli kill -a || true + fi +} + +cloudiatr_request_consent() { + local message="${*}" \ + response + + if "${CLOUDIATR_DONT_ASK_JUST_KISS}"; then + # ... the data goodbye. + return 0 + fi + + echo -n "cloudiatr: ${message} [y/n] " + # XXX: Don't use "read -p" as it may work unreliably + read response + + # "No" means "no". Everything but "y" also means "no". + [ "${response}" = "y" ] +} + +cloudiatr_has_eviction_consent() { + echo "cloudiatr (${CLOUDIATR_VERSION}) can't wait to evict '$(hostname)' ..." + echo + echo "Depending on your jurisdiction, 'eviction without consent' may be against the law." + echo "cloudiatr doesn't bother to make backups of the existing data. That's what zogftw is for." + echo + cloudiatr_request_consent "Continue eviction?" +} + +cloudiatr_evict() { + if cloudiatr_has_eviction_consent; then + if "${CLOUDIATR_MURDER_DEATH_KILL_REQUESTED}"; then + cloudiatr_murder_death_kill + else + cloudiatr_fyi "You asked for it ..." + fi + cloudiatr_fyi "Eviction in progress ..." + cloudiatr_evict_local_system + cloudiatr_fyi "Looks like somebody managed to install a real operating system ..." + if cloudiatr_request_consent "Reboot now?"; then + shutdown -r now + fi + return 0 + else + cloudiatr_fyi "Eviction aborted in time ..." + return 1 + fi +} + +cloudiatr_has_soft_protect_consent() { + cloudiatr_request_consent "Put $(uname) in 'Soviet Germany' mode?" +} + +# XXX: Only works for the cloudiatr disk layout. +cloudiatr_soft_protect() { + local \ + mirror_name device_to_clear number_of_disks + + mirror_name="vdev-remains" + number_of_disks="$(cloudiatr_get_number_of_disks)" + + cloudiatr_fyi "Destroying ${CLOUDIATR_BPOOL_NAME} ..." + cloudiatr_fyi "Use 'geli kill -a' to 'hard-protect' your data right now. No recovery without remote backups!" + + if zpool list "${CLOUDIATR_BPOOL_NAME}" >/dev/null 2>&1; then + zpool destroy "${CLOUDIATR_BPOOL_NAME}" || true + fi + + if [ "${number_of_disks}" = 1 ]; then + cloudiatr_fyi "Nuking former ${CLOUDIATR_BPOOL_NAME} vdev from orbit ..." + device_to_clear="$(cloudiatr_get_bpool_geoms)" + else + cloudiatr_fyi "Nuking former ${CLOUDIATR_BPOOL_NAME} vdevs from orbit using gmirror power ..." + gmirror load 2>/dev/null || true + gmirror label "${mirror_name}" $(cloudiatr_get_bpool_geoms) + device_to_clear="/dev/mirror/${mirror_name}" + fi + + geli onetime -s 4096 -e "${CLOUDIATR_GELI_EALGO}" "${device_to_clear}" + dd if=/dev/zero bs=1M of="${device_to_clear}.eli" 2>/dev/null || true + geli detach "${device_to_clear##/dev/}" + + cloudiatr_fyi "Done. Levelling nuked wasteland with zeroes ..." + dd if=/dev/zero bs=1M of="${device_to_clear}" 2>/dev/null || true + + if [ "${number_of_disks}" != 1 ]; then + gmirror destroy "${mirror_name}" + fi + + for device_to_clear in $(cloudiatr_get_bpool_geoms); do + cloudiatr_fyi "Done. Trying to trim device ${device_to_clear}. Failures will be ignored ..." + trim -f "${device_to_clear}" + done + + cloudiatr_fyi "Done. $(uname) should remain working as expected until the next shutdown ..." + cloudiatr_fyi "Remember to 'unprotect' the system before consensual reboots (or use the opportunity to test your backup system)" +} + +cloudiatr_check_privileges() { + local subcommand \ + unprivileged_subcommand uid user + + subcommand="${1}" + + for unprivileged_subcommand in ${CLOUDIATR_UNPRIVILEGED_SUBCOMMANDS}; do + if [ "${subcommand}" != "${unprivileged_subcommand}" ]; then + # subcommand does not require privileges + return 0 + fi + done + + uid="$(id -u)" + user="$(id -un)" + + if [ "${uid}" != 0 ]; then + cloudiatr_wtf "Check your privileges, $user. It looks like you might not have enough of them!" + cloudiatr_fyi "Hint: Using 'sudo' or 'su' might help." + return 1 + fi + + return 0 +} + +cloudiatr_usage() { + local \ + subcommand + + for subcommand in ${CLOUDIATR_SUBCOMMANDS}; do + echo "cloudiatr [-f config-file] $subcommand" + done +} + +cloudiatr_update_base() { + local \ + base_dist base_dbg_dist + + base_dist="${CLOUDIATR_DIST_DIR}/base.txz" + base_dbg_dist="${CLOUDIATR_DIST_DIR}/base-dbg.txz" + + if [ ! -f "${base_dist}" ]; then + cloudiatr_wtf "Base update impossible. '${base_dist}' does not exist" + return 1 + fi + cloudiatr_fyi "Updating base system ..." + chflags -R noschg /bin/ /lib/ /sbin/ /usr/bin/ /usr/lib /libexec/ || return 1 + cd / || return 1 + tar xpf "${base_dist}" \ + --exclude ./etc/ --exclude ./var/empty --exclude ./usr/src || return 1 + if [ -f "${base_dbg_dist}" ]; then + tar xpf "${base_dbg_dist}" || return 1 + fi +} + +cloudiatr_update_kernel() { + local \ + kernel_dist kernel_dbg_dist \ + kernel_dir kernel_dir_old \ + kernel_dbg_dir kernel_dbg_dir_old \ + + kernel_dist="${CLOUDIATR_DIST_DIR}/kernel.txz" + kernel_dbg_dist="${CLOUDIATR_DIST_DIR}/kernel-dbg.txz" + kernel_dir=/boot/kernel + kernel_dir_old=/boot/kernel.old + kernel_dbg_dir="/usr/lib/debug${kernel_dir}" + kernel_dbg_dir_old="/usr/lib/debug${kernel_dir_old}" + + if [ ! -f "${kernel_dist}" ]; then + cloudiatr_wtf "Kernel update impossible. '${kernel_dist}' does not exist" + return 1 + fi + cloudiatr_fyi "Updating kernel ..." + cd / || return 1 + if [ -d "${kernel_dir_old}" ]; then + cloudiatr_fyi "Removing ${kernel_dir_old}" + rm -r "${kernel_dir_old}" || return 1 + fi + cloudiatr_fyi "Moving ${kernel_dir} to ${kernel_dir_old} ..." + mv "${kernel_dir}" "${kernel_dir_old}" || return 1 + + cloudiatr_fyi "Extracting ${kernel_dist} ..." + tar xpf "${kernel_dist}" || return 1 + + if [ -d "${kernel_dbg_dir_old}" ]; then + cloudiatr_fyi "Removing ${kernel_dbg_dir_old}" + rm -r "${kernel_dbg_dir_old}" || return 1 + fi + if [ -d "${kernel_dbg_dir}" ]; then + cloudiatr_fyi "Moving ${kernel_dbg_dir} to ${kernel_dbg_dir_old} ..." + mv "${kernel_dbg_dir}" "${kernel_dbg_dir_old}" || return 1 + fi + cloudiatr_fyi "Extracting ${kernel_dbg_dist} ..." + if [ -f "${kernel_dbg_dist}" ]; then + tar xpf "${kernel_dbg_dist}" || return 1 + fi +} + +cloudiatr_show_version() { + echo "cloudiatr ${CLOUDIATR_VERSION}" +} + +cloudiatr_main() { + local mode + + if [ "${1}" = "-f" ]; then + shift + CLOUDIATR_CONFIG_FILE="${1}" + shift + if [ -z "${CLOUDIATR_CONFIG_FILE}" ] || + ! [ -f "${CLOUDIATR_CONFIG_FILE}" ]; then + cloudiatr_wtf "No existing config file specified." + return 1 + fi + fi + + mode="${1}" + + cloudiatr_check_privileges "${mode}" || return 1 + + cloudiatr_init "${mode}" + + case "${mode}" in + clean-up) + set +e + cloudiatr_clean_up + ;; + cmd) + shift + "${@}" + ;; + recreate-bpool) + cloudiatr_recreate_bpool "/" + ;; + evict) + cloudiatr_evict + ;; + soft-protect) + if cloudiatr_has_soft_protect_consent; then + cloudiatr_soft_protect + fi + ;; + update) + cloudiatr_update_base || return 1 + cloudiatr_install_bootcode "/boot" || return 1 + cloudiatr_update_kernel || return 1 + cloudiatr_create_bpool || return 1 + cloudiatr_populate_bpool "/" || return 1 + ;; + update-base) + cloudiatr_update_base || return 1 + ;; + update-bootcode) + cloudiatr_install_bootcode "/boot" || return 1 + ;; + update-kernel) + cloudiatr_update_kernel || return 1 + ;; + show-config) + cloudiatr_show_config + ;; + version) + cloudiatr_show_version + ;; + #rekey) + # XXX: Not yet implemented + *) + cloudiatr_usage + cloudiatr_wtf "Invalid or missing subcommand" + return 1 + ;; + esac + +} + +cloudiatr_main "${@}" -- 2.37.1 From d9391cfeef5be99ea607f1b7b319e9c4c6ee77fb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 4 Mar 2015 14:22:43 +0100 Subject: [PATCH 309/310] Allow to randomize MAC addresses at boot time through /etc/rc.conf Obtained from: ElectroBSD --- libexec/rc/Makefile | 2 +- libexec/rc/mac-randomization.subr | 76 +++++++++++++++++++++++++++++++ libexec/rc/rc.d/Makefile | 1 + libexec/rc/rc.d/netif | 3 ++ libexec/rc/rc.d/rether | 76 +++++++++++++++++++++++++++++++ 5 files changed, 157 insertions(+), 1 deletion(-) create mode 100755 libexec/rc/mac-randomization.subr create mode 100755 libexec/rc/rc.d/rether diff --git a/libexec/rc/Makefile b/libexec/rc/Makefile index 974eb8661182..7dd004896a23 100644 --- a/libexec/rc/Makefile +++ b/libexec/rc/Makefile @@ -4,7 +4,7 @@ CONFGROUPS= CONFETC CONFETCEXEC CONFETCDEFAULTS CONFETCDIR= /etc -CONFETC= network.subr rc rc.initdiskless rc.subr rc.shutdown rc.bsdextended +CONFETC= network.subr mac-randomization.subr rc rc.initdiskless rc.subr rc.shutdown rc.bsdextended CONFETCPACKAGE= rc .if ${MK_IPFW} != "no" diff --git a/libexec/rc/mac-randomization.subr b/libexec/rc/mac-randomization.subr new file mode 100755 index 000000000000..d486bf454591 --- /dev/null +++ b/libexec/rc/mac-randomization.subr @@ -0,0 +1,76 @@ +#!/bin/sh +# +########################################################################### +# +# Sub routines to randomizes MAC addresses after r287197 (WIP!) +# +# Add the following line to /etc/rc.conf to randomize the MAC +# address for all recognized network interfaces that got one +# at startup: +# +# rether_enable="YES" +# +# You can specify the interfaces manually like this: +# +# rether_interfaces="bge0 iwn0" +# +# Rether requires an ifconfig version that understands "ether random". +# +########################################################################### +# +# Copyright (c) 2014 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ALL YOUR +# DATA IS BELONG TO THE SOFTWARE AND MAY BE EATEN BY IT. IF THAT IS NOT +# ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. +########################################################################### + +rether_enable="${rether_enable-NO}" + +get_ethernet_interfaces() { + ifconfig -l ether +} + +get_ethernet_address() { + local interface="${1}" + + ifconfig "${interface}" ether | awk '/ether/ {print $2}' +} + +randomize_mac_addresses() { + local \ + interface \ + ethernet_address + + if [ "${rether_enable}" = "NO" ]; then + echo "Not randomizing any MAC addresses!" + return 0 + fi + + if [ -z "${rether_interfaces}" ]; then + rether_interfaces="$(get_ethernet_interfaces)" + fi + + echo "Randomizing MAC addresses for: ${rether_interfaces}" + + for interface in $rether_interfaces; + do + ethernet_address="$(get_ethernet_address "${interface}")" + + ifconfig "${interface}" ether random + + if [ "${ethernet_address}" = "$(get_ethernet_address "${interface}")" ]; then + echo "Failed to randomize MAC address for ${interface}: ${ethernet_address}" + fi + done +} diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index a9959e7e5534..694e904df8ab 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -82,6 +82,7 @@ CONFS= DAEMON \ rarpd \ rctl \ resolv \ + rether \ root \ route6d \ routing \ diff --git a/libexec/rc/rc.d/netif b/libexec/rc/rc.d/netif index b0d405b8cd45..bf7bfcee1353 100755 --- a/libexec/rc/rc.d/netif +++ b/libexec/rc/rc.d/netif @@ -32,6 +32,7 @@ . /etc/rc.subr . /etc/network.subr +. /etc/mac-randomization.subr name="netif" desc="Network interface setup" @@ -74,6 +75,8 @@ netif_start() # Create cloned interfaces clone_up $cmdifn + randomize_mac_addresses + # Rename interfaces. ifnet_rename $cmdifn diff --git a/libexec/rc/rc.d/rether b/libexec/rc/rc.d/rether new file mode 100755 index 000000000000..d486bf454591 --- /dev/null +++ b/libexec/rc/rc.d/rether @@ -0,0 +1,76 @@ +#!/bin/sh +# +########################################################################### +# +# Sub routines to randomizes MAC addresses after r287197 (WIP!) +# +# Add the following line to /etc/rc.conf to randomize the MAC +# address for all recognized network interfaces that got one +# at startup: +# +# rether_enable="YES" +# +# You can specify the interfaces manually like this: +# +# rether_interfaces="bge0 iwn0" +# +# Rether requires an ifconfig version that understands "ether random". +# +########################################################################### +# +# Copyright (c) 2014 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ALL YOUR +# DATA IS BELONG TO THE SOFTWARE AND MAY BE EATEN BY IT. IF THAT IS NOT +# ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. +########################################################################### + +rether_enable="${rether_enable-NO}" + +get_ethernet_interfaces() { + ifconfig -l ether +} + +get_ethernet_address() { + local interface="${1}" + + ifconfig "${interface}" ether | awk '/ether/ {print $2}' +} + +randomize_mac_addresses() { + local \ + interface \ + ethernet_address + + if [ "${rether_enable}" = "NO" ]; then + echo "Not randomizing any MAC addresses!" + return 0 + fi + + if [ -z "${rether_interfaces}" ]; then + rether_interfaces="$(get_ethernet_interfaces)" + fi + + echo "Randomizing MAC addresses for: ${rether_interfaces}" + + for interface in $rether_interfaces; + do + ethernet_address="$(get_ethernet_address "${interface}")" + + ifconfig "${interface}" ether random + + if [ "${ethernet_address}" = "$(get_ethernet_address "${interface}")" ]; then + echo "Failed to randomize MAC address for ${interface}: ${ethernet_address}" + fi + done +} -- 2.37.1 From 4390794a793183acfe324a1afb54084677d75b1a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 28 Aug 2022 17:38:00 +0200 Subject: [PATCH 310/310] Add reproduce.conf for ElectroBSD-20220822-d9391cfeef5b Expected checksums when reproducing on amd64: SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/base-dbg.txz) = 9938b04e363c5a191d8c6acc111a3bb22c47b3bf2a258577582c94c4c5c96061 SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/base.txz) = ab5b5fd06172e99678770c70605a75b31e46ea5455b1ff52c18849639c2069d9 SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/kernel-dbg.txz) = c472db227855c9bdbe3691038f12c0108208783e79cdb282567105b0b6fdc045 SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/kernel.txz) = b965fa28657a84b4128c8f124d090d883637b19a47d51a84e70cc77a708a5e78 SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/lib32-dbg.txz) = cbaca3748de08f91581992d8cd6cc2daf3c2425e51e379bb56b86d294ae41307 SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/lib32.txz) = 3f973dafbd4f77be4d2e216085fb4906d0eb79eef389a0e723aba952f905153b SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/MANIFEST) = abfbb9b82e4f3a938111ea4617f24b7e3f8137c177e7f1fb58b6c777659021f4 SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/src.txz) = df1130df1f0b7050b0202e4e7836394e918af1a31e5239ece5fabd316d2554ca SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/tests.txz) = e5a360c3300a06cb71ef5e75377c90fe0d95b4b8f5907b6bb06b08fb11157acf SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/lib32.txz) = 3f973dafbd4f77be4d2e216085fb4906d0eb79eef389a0e723aba952f905153b SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/src.txz) = df1130df1f0b7050b0202e4e7836394e918af1a31e5239ece5fabd316d2554ca SHA256 (ElectroBSD-20220822-d9391cfeef5b-j2/tests.txz) = e5a360c3300a06cb71ef5e75377c90fe0d95b4b8f5907b6bb06b08fb11157acf Obtained from: ElectroBSD --- reproduce.conf | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 reproduce.conf diff --git a/reproduce.conf b/reproduce.conf new file mode 100644 index 000000000000..e8d4302bde1d --- /dev/null +++ b/reproduce.conf @@ -0,0 +1,2 @@ +BUILD=ElectroBSD-20220822-d9391cfeef5b +EPOCH=1661151358 -- 2.37.1