From df95b8058ed993a362e14fe601e6488addc0daaa Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 10 Aug 2017 08:47:44 +0200 Subject: [PATCH 001/325] usb: Add UQ_MSC_NO_INQUIRY quirk for external "Intenso Memory" disk Without the quirk the disk disappears shortly after beeing detected: 08:36:35 t520 kernel: ugen1.4: at usbus1 08:36:36 t520 kernel: umass0 on uhub3 08:36:36 t520 kernel: umass0: on usbus1 08:36:36 t520 kernel: umass0: SCSI over Bulk-Only; quirks = 0x0100 08:36:36 t520 kernel: umass0:5:0: Attached to scbus5 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): REPORT LUNS. CDB: a0 00 00 00 00 00 00 00 00 10 00 00 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: SCSI Status Error 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): SCSI status: Check Condition 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code) 08:36:36 t520 kernel: (probe0:umass-sim0:0:0:0): Error 22, Unretryable error 08:37:49 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:37:49 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:37:49 t520 kernel: (probe0:umass-sim0:0:0:0): Retrying command 08:37:54 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:37:54 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:37:54 t520 kernel: (probe0:umass-sim0:0:0:0): Retrying command 08:38:00 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:38:00 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:38:00 t520 kernel: (probe0:umass-sim0:0:0:0): Retrying command 08:38:05 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:38:05 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:38:05 t520 kernel: (probe0:umass-sim0:0:0:0): Retrying command 08:38:11 t520 kernel: (probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 01 83 00 fc 00 08:38:11 t520 kernel: (probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error 08:38:11 t520 kernel: (probe0:umass-sim0:0:0:0): Error 5, Retries exhausted 08:38:44 t520 kernel: (da0:umass-sim0:0:0:0): got CAM status 0x44 08:38:44 t520 kernel: (da0:umass-sim0:0:0:0): fatal error, failed to attach to device 08:38:44 t520 kernel: g_access(918): provider da0 has error 6 set 08:38:44 t520 last message repeated 2 times fk@t520 ~ $sudo usbconfig -u 1 -a 4 dump_device_desc ugen1.4: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (2mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0210 bDeviceClass = 0x0000 bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0040 idVendor = 0x2109 idProduct = 0x0701 bcdDevice = 0x0307 iManufacturer = 0x0001 iProduct = 0x0002 iSerialNumber = 0x0003 <20130519000569> bNumConfigurations = 0x0001 PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221852 PR submission date: 2017-08-27 Obtained from: ElectroBSD --- sys/dev/usb/quirk/usb_quirk.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sys/dev/usb/quirk/usb_quirk.c b/sys/dev/usb/quirk/usb_quirk.c index b9d277692137..12164094568e 100644 --- a/sys/dev/usb/quirk/usb_quirk.c +++ b/sys/dev/usb/quirk/usb_quirk.c @@ -270,6 +270,7 @@ static struct usb_quirk_entry usb_quirks[USB_DEV_QUIRKS_MAX] = { USB_QUIRK(INSYSTEM, USBCABLE, 0x0000, 0xffff, UQ_MSC_FORCE_WIRE_CBI, UQ_MSC_FORCE_PROTO_ATAPI, UQ_MSC_NO_TEST_UNIT_READY, UQ_MSC_NO_START_STOP, UQ_MSC_ALT_IFACE_1), + USB_QUIRK(INTENSO, MEMORY_BOX, 0x0000, 0xffff, UQ_MSC_NO_INQUIRY), USB_QUIRK(INSYSTEM, ATAPI, 0x0000, 0xffff, UQ_MSC_FORCE_WIRE_CBI, UQ_MSC_FORCE_PROTO_RBC), USB_QUIRK(INSYSTEM, STORAGE_V2, 0x0000, 0xffff, UQ_MSC_FORCE_WIRE_CBI, -- 2.32.0 From 628b0758b7c0a7d5a029d140be91d7cc44ebbe43 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 3 Jul 2015 09:54:11 +0200 Subject: [PATCH 002/325] g_new_provider_event(): Return early if the provider is already withering Previously debug kernels would panic, other side effects haven't been diagnosed yet. Patch by Scott M. Ferris. FreeBSD bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200740 Obtained from: ElectroBSD --- sys/geom/geom_subr.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/geom/geom_subr.c b/sys/geom/geom_subr.c index ccc048fe865d..3d95f717b832 100644 --- a/sys/geom/geom_subr.c +++ b/sys/geom/geom_subr.c @@ -540,8 +540,8 @@ g_new_provider_event(void *arg, int flag) return; pp = arg; G_VALID_PROVIDER(pp); - KASSERT(!(pp->flags & G_PF_WITHER), - ("g_new_provider_event but withered")); + if ((pp->flags & G_PF_WITHER) != 0) + return; LIST_FOREACH_SAFE(cp, &pp->consumers, consumers, next_cp) { if ((cp->flags & G_CF_ORPHAN) == 0 && cp->geom->attrchanged != NULL) -- 2.32.0 From 2dfaf6d1427c8c7f1b108a434ecf94c860bd6c37 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 21 May 2017 14:09:13 +0200 Subject: [PATCH 003/325] libdtrace: Prevent an assertion from triggering with certain llquantize() parameters The dtrace command that reached the assertion: fk@t520 ~ $sudo dtrace -n 'syscall::read:return /execname == "privoxy"/ { @[execname] = llquantize(arg0, 100, 0, 10, 100); @m = max(arg0)}' [...] 9800 | 0 9900 | 0 10000 |@@@@@@@@@@@@@@@@@@@@ 37 20000 | 0 Assertion failed: (value < next), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c, line 1083. Abort trap (gdb) where #0 0x00000008011effda in thr_kill () from /lib/libc.so.7 #1 0x00000008011effa4 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52 #2 0x00000008011eff19 in abort () at /usr/src/lib/libc/stdlib/abort.c:65 #3 0x000000080088c3b2 in __assert (expr=0x8008d3172 "value < next", file=0x8008d3078 "/usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c", line=1083) at /usr/src/cddl/lib/libdtrace/../../../cddl/compat/opensolaris/include/assert.h:56 #4 0x000000080088c190 in dt_print_llquantize (dtp=0x802633000, fp=0x8014c37e8, addr=0x80269a110, size=7840, normal=1) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:1083 #5 0x000000080088e37d in dt_print_datum (dtp=0x802633000, fp=0x8014c37e8, rec=0x8026900e8, addr=0x80269a110 "d", size=7848, aggdata=0x802690150, normal=1, pd=0x7fffffffe750) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:2211 #6 0x000000080088dc12 in dt_print_aggs (aggsdata=0x7fffffffe630, naggvars=1, arg=0x7fffffffe750) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:2313 #7 0x000000080088e6cf in dt_print_agg (aggdata=0x802690150, arg=0x7fffffffe750) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:2361 #8 0x0000000800895f8b in dt_aggregate_walk_sorted (dtp=0x802633000, func=0x80088e610 , arg=0x7fffffffe750, sfunc=0x0) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_aggregate.c:1585 #9 0x0000000800895d39 in dtrace_aggregate_walk_sorted (dtp=0x802633000, func=0x80088e610 , arg=0x7fffffffe750) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_aggregate.c:1605 #10 0x0000000800897f12 in dtrace_aggregate_print (dtp=0x802633000, fp=0x8014c37e8, func=0x800895d10 ) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_aggregate.c:2130 #11 0x0000000000403a5e in main (argc=, argv=) at /usr/src/cddl/usr.sbin/dtrace/../../../cddl/contrib/opensolaris/cmd/dtrace/dtrace.c:2005 (gdb) f 4 #4 0x000000080088c190 in dt_print_llquantize (dtp=0x802633000, fp=0x8014c37e8, addr=0x80269a110, size=7840, normal=1) at /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c:1083 1083 assert(value < next); (gdb) p step $2915 = 77662796314522419 (gdb) p value $2916 = 7834326075677972872 (gdb) p next $2917 = 7766279631452241920 It works as expected when replacing the 10 with a 5. Various other parameter combinations work as expected as well and I've used similar commands for weeks without issues. The problem is reproducible with other execnames as long as the probe fires. The "@m = max(arg0)" part isn't required to trigger the assertion but I only noticed it after already patching the system where libdtrace is build with reduced optimizations. This commit may not be the best solution but it seems to reliably prevent the core dumps and the output continues to look reasonable. The code flow in dt_print_llquantize() seems strange to me and maybe the loop should break if "bin" reaches "last_bin" instead. My impression is that it does a bunch of cycles at the end without doing meaningful work. PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219451 PR submission date: 2017-05-22 Superior upstream commit in HEAD: r322773/95f65483b371f Obtained from: ElectroBSD --- cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c index 62c120388b18..51d7f332a28d 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c @@ -1084,7 +1084,7 @@ dt_print_llquantize(dtrace_hdl_t *dtp, FILE *fp, const void *addr, assert(value < next); bin++; - if ((value += step) != next) + if ((value += step) < next) continue; next = value * factor; -- 2.32.0 From b1adc706434c55f39e1b3157445d59f0f8a45c9e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 23 Aug 2017 08:52:11 +0200 Subject: [PATCH 004/325] share/dtrace: Stop pretending that the DTraceToolkit scripts can be installed ... by setting MK_CDDL. The DTraceToolkit scripts were removed in r300226 and now have to be installed through other means. Obtained from: ElectroBSD --- share/dtrace/Makefile | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 95e0b4fee4e3..c3ec2d9ce510 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -1,7 +1,6 @@ # $FreeBSD$ # -# Hand installing our scripts and optionally (based on MK_CDDL) installing -# the DTraceToolkit. +# Hand installing our scripts. # .include -- 2.32.0 From 48fa67732d9da45f0fd35f9b46c1ca09475eaf54 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 20 Apr 2017 20:07:58 +0200 Subject: [PATCH 005/325] sys/cddl: Add a sysctl to toggle zfs_free_leak_on_eio Setting the sysctl allows to break a previously-endless cycle of ZFS collecting checksum errors for metadata. Before setting vfs.zfs.free_leak_on_eio=1: fk@t520 ~ $zpool status cloudia2 pool: cloudia2 state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://illumos.org/msg/ZFS-8000-8A scan: scrub repaired 308K in 53h23m with 3358 errors on Sun Apr 16 20:33:26 2017 config: NAME STATE READ WRITE CKSUM cloudia2 ONLINE 0 0 129 label/cloudia2.eli ONLINE 0 0 516 errors: 3362 data errors, use '-v' for a list fk@t520 ~ $zpool status -v cloudia2 [..] errors: Permanent errors have been detected in the following files: <0x186>:<0x28> <0x186>:<0x35> <0xffffffffffffffff>:<0x28> Every five seconds the checksum counter got increased. zfsdbg-msg output: 2017 Apr 21 11:12:43: bptree index 0: traversing from min_txg=1 bookmark -1/40/0/5120 2017 Apr 21 11:12:43: bptree index 1: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:43: bptree index 2: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:43: bptree index 3: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:43: bptree index 4: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:43: bptree index 5: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:43: bptree index 6: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:43: bptree index 7: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:48: bptree index 0: traversing from min_txg=1 bookmark -1/40/0/5120 2017 Apr 21 11:12:48: bptree index 1: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:48: bptree index 2: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:48: bptree index 3: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:48: bptree index 4: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:48: bptree index 5: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:48: bptree index 6: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:48: bptree index 7: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:53: bptree index 0: traversing from min_txg=1 bookmark -1/40/0/5120 2017 Apr 21 11:12:53: bptree index 1: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:53: bptree index 2: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:53: bptree index 3: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:53: bptree index 4: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:53: bptree index 5: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:53: bptree index 6: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:53: bptree index 7: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:58: bptree index 0: traversing from min_txg=1 bookmark -1/40/0/5120 2017 Apr 21 11:12:58: bptree index 1: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:58: bptree index 2: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:58: bptree index 3: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:58: bptree index 4: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:58: bptree index 5: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:58: bptree index 6: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:12:58: bptree index 7: traversing from min_txg=-1 bookmark 0/0/0/0 fk@t520 ~ $zpool get all cloudia2 NAME PROPERTY VALUE SOURCE cloudia2 size 2.98T - cloudia2 capacity 54% - cloudia2 altroot - default cloudia2 health ONLINE - cloudia2 guid 4205907112567218706 default cloudia2 version - default cloudia2 bootfs - default cloudia2 delegation on default cloudia2 autoreplace off default cloudia2 cachefile - default cloudia2 failmode wait default cloudia2 listsnapshots off default cloudia2 autoexpand off default cloudia2 dedupditto 0 default cloudia2 dedupratio 1.00x - cloudia2 free 1.37T - cloudia2 allocated 1.62T - cloudia2 readonly off - cloudia2 comment - default cloudia2 expandsize - - cloudia2 freeing 24.2G default cloudia2 fragmentation 32% - cloudia2 leaked 0 default [...] After setting vfs.zfs.free_leak_on_eio=1: zfsdbg-msg output: 2017 Apr 21 11:13:03: bptree index 0: traversing from min_txg=1 bookmark -1/40/0/5120 2017 Apr 21 11:13:06: freed 100000 blocks in 3050ms from free_bpobj/bptree txg 17892; err=-1 2017 Apr 21 11:13:07: bptree index 0: traversing from min_txg=1 bookmark -1/68/0/718 2017 Apr 21 11:13:08: bptree index 1: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:13:08: bptree index 2: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:13:08: bptree index 3: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:13:08: bptree index 4: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:13:08: bptree index 5: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:13:08: bptree index 6: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:13:08: bptree index 7: traversing from min_txg=-1 bookmark 0/0/0/0 2017 Apr 21 11:13:08: freed 96110 blocks in 1927ms from free_bpobj/bptree txg 17893; err=0 2017 Apr 21 11:15:33: command: zpool clear cloudia2 The checksum error counters stopped incrementing, "freeing" went to 0 and "leaked" from 0 to 256M. fk@t520 ~ $zpool get all cloudia2 NAME PROPERTY VALUE SOURCE cloudia2 size 2.98T - cloudia2 capacity 53% - cloudia2 altroot - default cloudia2 health ONLINE - cloudia2 guid 4205907112567218706 default cloudia2 version - default cloudia2 bootfs - default cloudia2 delegation on default cloudia2 autoreplace off default cloudia2 cachefile - default cloudia2 failmode wait default cloudia2 listsnapshots off default cloudia2 autoexpand off default cloudia2 dedupditto 0 default cloudia2 dedupratio 1.00x - cloudia2 free 1.39T - cloudia2 allocated 1.59T - cloudia2 readonly off - cloudia2 comment - default cloudia2 expandsize - - cloudia2 freeing 0 default cloudia2 fragmentation 32% - cloudia2 leaked 256M default [...] The difference on the receiving side confirmed that some space had been recovered: [fk@kendra ~]$ zfs list -r -p -t all dpool/ggated/cloudia2 NAME USED AVAIL REFER MOUNTPOINT [...] dpool/ggated/cloudia2@2017-04-21_10:37 9251840 - 1812645106176 - dpool/ggated/cloudia2@2017-04-21_11:17 3950592 - 1800267106304 - It's not obvious to me if the 256M were really leaked but either way it looks like a clear win. On another ZFS pool with the same issue but backed by an USB disk all the space in "freeing" was supposedly "leaked" but it was a lot less to begin with: Before setting vfs.zfs.free_leak_on_eio=1: fk@t520 /usr/src $zpool get all wde4 NAME PROPERTY VALUE SOURCE wde4 size 1.81T - wde4 capacity 94% - wde4 altroot - default wde4 health ONLINE - wde4 guid 14402430966328721211 default wde4 version - default wde4 bootfs - default wde4 delegation on default wde4 autoreplace off default wde4 cachefile - default wde4 failmode wait default wde4 listsnapshots off default wde4 autoexpand off default wde4 dedupditto 0 default wde4 dedupratio 1.00x - wde4 free 107G - wde4 allocated 1.71T - wde4 readonly off - wde4 comment - default wde4 expandsize - - wde4 freeing 1.18M default wde4 fragmentation 23% - wde4 leaked 0 default After setting vfs.zfs.free_leak_on_eio=1: fk@t520 /usr/src $zpool get all wde4 NAME PROPERTY VALUE SOURCE wde4 size 1.81T - wde4 capacity 94% - wde4 altroot - default wde4 health ONLINE - wde4 guid 14402430966328721211 default wde4 version - default wde4 bootfs - default wde4 delegation on default wde4 autoreplace off default wde4 cachefile - default wde4 failmode wait default wde4 listsnapshots off default wde4 autoexpand off default wde4 dedupditto 0 default wde4 dedupratio 1.00x - wde4 free 107G - wde4 allocated 1.71T - wde4 readonly off - wde4 comment - default wde4 expandsize - - wde4 freeing 0 default wde4 fragmentation 23% - wde4 leaked 1.18M default [...] The pool was affected by the issue since 2015: https://lists.freebsd.org/pipermail/freebsd-fs/2015-February/020845.html PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218954 PR submission date: 2017-04-29 12:19 UTC Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c index 0706767a9d1f..8671e7b38923 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c @@ -371,6 +371,8 @@ SYSCTL_INT(_vfs_zfs, OID_AUTO, deadman_enabled, CTLFLAG_RWTUN, &zfs_deadman_enabled, 0, "Kernel panic on stalled ZFS I/O"); SYSCTL_INT(_vfs_zfs, OID_AUTO, spa_asize_inflation, CTLFLAG_RWTUN, &spa_asize_inflation, 0, "Worst case inflation factor for single sector writes"); +SYSCTL_INT(_vfs_zfs, OID_AUTO, free_leak_on_eio, CTLFLAG_RWTUN, + &zfs_free_leak_on_eio, 0, "Leak space from indirect blocks that can not be read"); #endif #ifndef illumos -- 2.32.0 From 15c5c3191047b457aa71aaa1530bc8cc87eb511a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 4 Jun 2017 13:22:42 +0200 Subject: [PATCH 006/325] sys/cddl: Add a sysctl to toggle send_corrupt_data Enabling it allows to send datasets with corrupted blocks which is useful to recover data from pools with dying disks. Blocks filled with 0x'zfs badd bloc' are sent instead of the corrupted data. As a result, the receiving side may end up with more corrupt data than the sending side. While it would be preferable to send the corrupt data as is (assuming the block can be read but contains flipped bits), this would probably have to happen at a different layer and currently isn't done. The ZFSOnLinux people already added an option for this in 2013: https://github.com/zfsonlinux/zfs/issues/1982 Usage example: fk@t520 ~ $sudo zpool status -v wde2 pool: wde2 state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://illumos.org/msg/ZFS-8000-8A scan: scrub repaired 0 in 11h40m with 10 errors on Sun Jan 1 10:25:26 2017 config: NAME STATE READ WRITE CKSUM wde2 ONLINE 0 0 0 label/wde2.eli ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: wde2/backup/t520/tank/home/fk@2011-07-28_04:54:/Mail/Tor/Read/15654 wde2/backup/t520/tank/home/fk@2011-07-28_04:54:/Mail/Tor/Read/16411 [...] fk@t520 ~ $cat /wde2/backup/t520/tank/home/fk/.zfs/snapshot/2011-07-28_04\:54/Mail/Tor/Read/16411 cat: /wde2/backup/t520/tank/home/fk/.zfs/snapshot/2011-07-28_04:54/Mail/Tor/Read/16411: Input/output error fk@t520 ~ $dd if=/wde2/backup/t520/tank/home/fk/.zfs/snapshot/2011-07-28_04\:54/Mail/Tor/Read/16411 bs=1 dd: /wde2/backup/t520/tank/home/fk/.zfs/snapshot/2011-07-28_04:54/Mail/Tor/Read/16411: Input/output error 0+0 records in 0+0 records out 0 bytes transferred in 0.026960 secs (0 bytes/sec) fk@t520 ~ $sudo zfs send wde2/backup/t520/tank/home/fk@2011-07-28_04:54 | mbuffer | sudo zfs receive -v tank/corruption-test receiving full stream of wde2/backup/t520/tank/home/fk@2011-07-28_04:54 into tank/corruption-test@2011-07-28_04:54 in @ 0.0 KiB/s, out @ 0.0 KiB/s, 1178 MiB total, buffer 0% fullwarning: cannot send 'wde2/backup/t520/tank/home/fk@2011-07-28_04:54': Input/output error summary: 1178 MiByte in 5min 44.0sec - average of 3508 KiB/s cannot receive new filesystem stream: checksum mismatch or incomplete stream Toggling vfs.zfs.send_corrupt_data allows to send the whole snapshot with the corrupted data: fk@t520 ~ $sudo sysctl vfs.zfs.send_corrupt_data=1 vfs.zfs.send_corrupt_data: 0 -> 1 fk@t520 ~ $sudo zfs send wde2/backup/t520/tank/home/fk@2011-07-28_04:54 | mbuffer | sudo zfs receive -v tank/corruption-test receiving full stream of wde2/backup/t520/tank/home/fk@2011-07-28_04:54 into tank/corruption-test@2011-07-28_04:54 in @ 7193 KiB/s, out @ 7193 KiB/s, 1238 MiB total, buffer 0% full summary: 1239 MiByte in 43.6sec - average of 28.4 MiB/s received 1.21GB stream in 59 seconds (21.0MB/sec) fk@t520 ~ $sudo sysctl vfs.zfs.send_corrupt_data=0 vfs.zfs.send_corrupt_data: 1 -> 0 On the receiving side the corrupted block now has a valid checksum, the 0x'zfs badd bloc' pattern isn't obvious from userland and from ZFS's point of view the data is legit (which doesn't seem ideal either): fk@t520 ~ $hd /tank/corruption-test/Mail/Tor/Read/15654 00000000 0c b1 dd ba f5 02 00 00 0c b1 dd ba f5 02 00 00 |................| * 00001170 fk@t520 ~ $sudo zfs send tank/corruption-test@2011-07-28_04:54 | dd of=/dev/null bs=1m 0+198030 records in 1238+1 records out 1298720968 bytes transferred in 1.788083 secs (726320368 bytes/sec) PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221909 PR submission date: 2017-08-29 Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c index f4dcc4bcb976..1a7ba92fd9b3 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c @@ -75,6 +75,11 @@ int zfs_send_set_freerecords_bit = B_TRUE; #ifdef _KERNEL TUNABLE_INT("vfs.zfs.send_set_freerecords_bit", &zfs_send_set_freerecords_bit); + +SYSCTL_DECL(_vfs_zfs); +SYSCTL_INT(_vfs_zfs, OID_AUTO, send_corrupt_data, CTLFLAG_RWTUN, + &zfs_send_corrupt_data, 0, "Allow to send datasets with corrupt blocks. " + "Blocks filled with 0x'zfs badd bloc' are send as replacement blocks."); #endif static char *dmu_recv_tag = "dmu_recv_tag"; -- 2.32.0 From 1369195c1b1e0b2c45520ba814e5e6f7e602d8ff Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 11 May 2016 12:41:12 +0200 Subject: [PATCH 007/325] ZFS: Attempt to let the spa deadman store the last and the maximum vdev sync delay As it's useful to know when tuning the timeout. XXX: This looks like it should work, but actually doesn't. Obtained from: ElectroBSD --- .../contrib/opensolaris/uts/common/fs/zfs/vdev.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c index c91551062d3b..2b2dc374b068 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c @@ -98,6 +98,16 @@ SYSCTL_NODE(_vfs_zfs, OID_AUTO, vdev, CTLFLAG_RW, 0, "ZFS VDEV"); static uint64_t zfs_max_auto_ashift = SPA_MAXASHIFT; static uint64_t zfs_min_auto_ashift = SPA_MINASHIFT; +static uint64_t zfs_max_vdev_sync_delay = 0; +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, vdev_sync_delay_max, CTLFLAG_RWTUN, + &zfs_max_vdev_sync_delay, 0, + "Maximum vdev sync delay seen by the spa deadman"); + +static uint64_t zfs_last_vdev_sync_delay = 0; +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, vdev_sync_delay_last, CTLFLAG_RWTUN, + &zfs_last_vdev_sync_delay, 0, + "Last vdev sync delay seen seen by the spa deadman"); + static int sysctl_vfs_zfs_max_auto_ashift(SYSCTL_HANDLER_ARGS) { @@ -4502,6 +4512,9 @@ vdev_deadman(vdev_t *vd) */ fio = avl_first(&vq->vq_active_tree); delta = gethrtime() - fio->io_timestamp; + zfs_last_vdev_sync_delay = delta; + if (zfs_max_vdev_sync_delay < delta) + zfs_max_vdev_sync_delay = delta; if (delta > spa_deadman_synctime(spa)) { vdev_dbgmsg(vd, "SLOW IO: zio timestamp " "%lluns, delta %lluns, last io %lluns", -- 2.32.0 From f1ef3a90524e751fc3903e71c1c8ec12e238ec87 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 28 Jul 2015 15:07:52 +0200 Subject: [PATCH 008/325] etc/rc.d/dumpon: Get dumpdev=AUTO working for GEOM_ELI consumers Obtained from: ElectroBSD --- libexec/rc/rc.d/dumpon | 1 + 1 file changed, 1 insertion(+) diff --git a/libexec/rc/rc.d/dumpon b/libexec/rc/rc.d/dumpon index dddbf2af01cc..f3047b50dc03 100755 --- a/libexec/rc/rc.d/dumpon +++ b/libexec/rc/rc.d/dumpon @@ -49,6 +49,7 @@ dumpon_start() fi while read dev mp type more ; do [ "${type}" = "swap" ] || continue + dev="${dev%%.eli}" [ -c "${dev}" ] || continue dumpon_try "${dev}" 2>/dev/null && return 0 done Date: Sun, 19 Mar 2017 17:39:06 +0100 Subject: [PATCH 009/325] contrib/mtree: Fix small memory leak in spec() ... that was found when using makefs with an mtree spec: [fk@electrobsd /tmp/image-comparison] $valgrind --leak-check=full --tool=memcheck makefs -T 0 -B little -o label=blafasel -F mtree.spec image.part . ==35506== Memcheck, a memory error detector ==35506== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==35506== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info ==35506== Command: makefs -T 0 -B little -o label=blafasel -F mtree.spec image.part . ==35506== Calculated size of `image.part': 1917493248 bytes, 15 inodes Extent size set to 8192 density reduced from 127832884 to 907060 image.part: 1828.7MB (3745104 sectors) block size 8192, fragment size 1024 using 34 cylinder groups of 55.31MB, 7080 blks, 64 inodes. super-block backups (for fsck -b #) at: 32, 113312, 226592, 339872, 453152, 566432, 679712, 792992, 906272, 1019552, 1132832, 1246112, 1359392, 1472672, 1585952, 1699232, 1812512, 1925792, 2039072, 2152352, 2265632, 2378912, 2492192, 2605472, 2718752, 2832032, 2945312, 3058592, 3171872, 3285152, 3398432, 3511712, 3624992, 3738272, Populating `image.part' Image `image.part' complete ==35506== ==35506== HEAP SUMMARY: ==35506== in use at exit: 1,137,370 bytes in 271 blocks ==35506== total heap usage: 433,879 allocs, 433,608 frees, 1,792,473,254 bytes allocated ==35506== ==35506== 62 bytes in 1 blocks are definitely lost in loss record 1 of 15 ==35506== at 0x4C25DBE: realloc (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so) ==35506== by 0x41760A: spec (in /usr/sbin/makefs) ==35506== by 0x40BD90: apply_specfile (in /usr/sbin/makefs) ==35506== by 0x4090A7: main (in /usr/sbin/makefs) ==35506== ==35506== LEAK SUMMARY: ==35506== definitely lost: 62 bytes in 1 blocks ==35506== indirectly lost: 0 bytes in 0 blocks ==35506== possibly lost: 0 bytes in 0 blocks ==35506== still reachable: 1,137,308 bytes in 270 blocks ==35506== suppressed: 0 bytes in 0 blocks ==35506== Reachable blocks (those to which a pointer was found) are not shown. ==35506== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==35506== ==35506== For counts of detected and suppressed errors, rerun with: -v ==35506== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) Obtained from: ElectroBSD --- contrib/mtree/spec.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/mtree/spec.c b/contrib/mtree/spec.c index 6fbe6f534560..25ad285aec00 100644 --- a/contrib/mtree/spec.c +++ b/contrib/mtree/spec.c @@ -117,7 +117,7 @@ spec(FILE *fp) root = NULL; centry = last = NULL; - tname = NULL; + ntname = tname = NULL; tnamelen = 0; memset(&ginfo, 0, sizeof(ginfo)); for (mtree_lineno = 0; @@ -261,6 +261,7 @@ noparent: mtree_err("no parent node"); last = centry; } } + free(ntname); return (root); } -- 2.32.0 From 907058155290a2ccfab153378e98e684bc92e64c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 23 Feb 2015 11:38:09 +0100 Subject: [PATCH 010/325] sys/conf/newvers.sh: Set TYPE to ElectroBSD Obtained from: ElectroBSD --- sys/conf/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) mode change 100644 => 100755 sys/conf/newvers.sh diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh old mode 100644 new mode 100755 index 751d395ad519..6ec12a8958af --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -47,7 +47,7 @@ # Note: usr.sbin/amd/include/newvers.sh assumes all variable assignments of # upper case variables starting in column 1 are on one line w/o continuation. -TYPE="FreeBSD" +TYPE="ElectroBSD" REVISION="12.3" BRANCH="STABLE" if [ -n "${BRANCH_OVERRIDE}" ]; then -- 2.32.0 From d3e65df86feb867943ba11c306fd32a1bc61399f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Sep 2015 14:34:45 +0200 Subject: [PATCH 011/325] sys/conf/newvers.sh: Do not add git hash derived from .git if KERNEL_VERSION_NUMBER is defined ... as this indicates that we are (trying to) reproduce a build. Do not use the REPRO_SEED for this as it's always set nowadays. XXX: There probably is prettier solution to do this. Obtained from: ElectroBSD --- sys/conf/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index 6ec12a8958af..663237787c36 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -175,7 +175,7 @@ for dir in /usr/bin /usr/local/bin; do fi done -if findvcs .git; then +if [ -z "${KERNEL_VERSION_NUMBER}" ] && findvcs .git; then for dir in /usr/bin /usr/local/bin; do if [ -x "${dir}/git" ] ; then git_cmd="${dir}/git -c help.autocorrect=0 --git-dir=${VCSDIR}" -- 2.32.0 From 245ef076c7ebf0f64ab5330a0b26a6cda86e2990 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 28 May 2015 15:04:48 +0200 Subject: [PATCH 012/325] sys/conf/newvers.sh: Allow to overwrite the kernel version ... as the object directory may be dirty. Obtained from: ElectroBSD --- sys/conf/newvers.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index 663237787c36..bc3a496106ec 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -129,7 +129,7 @@ then fi touch version -v=`cat version` +v=${KERNEL_VERSION_NUMBER:-`cat version`} u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} @@ -295,4 +295,6 @@ int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF -echo $((v + 1)) > version +if [ -z "${KERNEL_VERSION_NUMBER}" ]; then + echo $((v + 1)) > version +fi -- 2.32.0 From 34bd904fce54ba306fc87a20556835b6581800ad Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2015 17:24:47 +0200 Subject: [PATCH 013/325] amd64: Save a copy of GENERIC as ELECTRO_BLOAT Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 365 +++++++++++++++++++++++++++++++++++ 1 file changed, 365 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BLOAT diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT new file mode 100644 index 000000000000..3d2cc39c6353 --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -0,0 +1,365 @@ +# +# GENERIC -- Generic kernel configuration file for FreeBSD/amd64 +# +# For more information on this file, please read the config(5) manual page, +# and/or the handbook section on Kernel Configuration Files: +# +# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html +# +# The handbook is also available locally in /usr/share/doc/handbook +# if you've installed the doc distribution, otherwise always see the +# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the +# latest information. +# +# An exhaustive list of options and more detailed explanations of the +# device lines is also present in the ../../conf/NOTES and NOTES files. +# If you are in doubt as to the purpose or necessity of a line, check first +# in NOTES. +# +# $FreeBSD$ + +cpu HAMMER +ident GENERIC + +makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols +makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support + +options SCHED_ULE # ULE scheduler +options PREEMPTION # Enable kernel thread preemption +options INET # InterNETworking +options INET6 # IPv6 communications protocols +options TCP_OFFLOAD # TCP offload +options SCTP # Stream Control Transmission Protocol +options FFS # Berkeley Fast Filesystem +options SOFTUPDATES # Enable FFS soft updates support +options UFS_ACL # Support for access control lists +options UFS_DIRHASH # Improve performance on big directories +options UFS_GJOURNAL # Enable gjournal-based UFS journaling +options QUOTA # Enable disk quotas for UFS +options MD_ROOT # MD is a potential root device +options NFSCL # Network Filesystem Client +options NFSD # Network Filesystem Server +options NFSLOCKD # Network Lock Manager +options NFS_ROOT # NFS usable as /, requires NFSCL +options MSDOSFS # MSDOS Filesystem +options CD9660 # ISO 9660 Filesystem +options PROCFS # Process filesystem (requires PSEUDOFS) +options PSEUDOFS # Pseudo-filesystem framework +options GEOM_PART_GPT # GUID Partition Tables. +options GEOM_RAID # Soft RAID functionality. +options GEOM_LABEL # Provides labelization +options COMPAT_FREEBSD32 # Compatible with i386 binaries +options COMPAT_FREEBSD4 # Compatible with FreeBSD4 +options COMPAT_FREEBSD5 # Compatible with FreeBSD5 +options COMPAT_FREEBSD6 # Compatible with FreeBSD6 +options COMPAT_FREEBSD7 # Compatible with FreeBSD7 +options COMPAT_FREEBSD9 # Compatible with FreeBSD9 +options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI +options KTRACE # ktrace(1) support +options STACK # stack(9) support +options SYSVSHM # SYSV-style shared memory +options SYSVMSG # SYSV-style message queues +options SYSVSEM # SYSV-style semaphores +options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions +options PRINTF_BUFR_SIZE=128 # Prevent printf output being interspersed. +options KBD_INSTALL_CDEV # install a CDEV entry in /dev +options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(4) +options AUDIT # Security event auditing +options CAPABILITY_MODE # Capsicum capability mode +options CAPABILITIES # Capsicum capabilities +options MAC # TrustedBSD MAC Framework +options KDTRACE_FRAME # Ensure frames are compiled in +options KDTRACE_HOOKS # Kernel DTrace hooks +options DDB_CTF # Kernel ELF linker loads CTF data +options INCLUDE_CONFIG_FILE # Include this file in kernel +options RACCT # Resource accounting framework +options RACCT_DEFAULT_TO_DISABLED # Set kern.racct.enable=0 by default +options RCTL # Resource limits + +# Debugging support. Always need this: +options KDB # Enable kernel debugger support. +options KDB_TRACE # Print a stack trace for a panic. +# For full debugger support use (turn off in stable branch): +options DDB # Support DDB. +options GDB # Support remote GDB. +options DEADLKRES # Enable the deadlock resolver +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones + +# Make an SMP-capable kernel by default +options SMP # Symmetric MultiProcessor Kernel + +# CPU frequency control +device cpufreq + +# Bus support. +device acpi +options ACPI_DMAR +device pci +options PCI_IOV # PCI SR-IOV support + +# Floppy drives +device fdc + +# ATA controllers +device ahci # AHCI-compatible SATA controllers +device ata # Legacy ATA/SATA controllers +options ATA_STATIC_ID # Static device numbering +device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA +device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA + +# SCSI Controllers +device ahc # AHA2940 and onboard AIC7xxx devices +options AHC_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~128k to driver. +device ahd # AHA39320/29320 and onboard AIC79xx devices +options AHD_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~215k to driver. +device esp # AMD Am53C974 (Tekram DC-390(T)) +device hptiop # Highpoint RocketRaid 3xxx series +device isp # Qlogic family +#device ispfw # Firmware for QLogic HBAs- normally a module +device mpt # LSI-Logic MPT-Fusion +device mps # LSI-Logic MPT-Fusion 2 +device mpr # LSI-Logic MPT-Fusion 3 +#device ncr # NCR/Symbios Logic +device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') +device trm # Tekram DC395U/UW/F DC315U adapters + +device adv # Advansys SCSI adapters +device adw # Advansys wide SCSI adapters +device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. +device bt # Buslogic/Mylex MultiMaster SCSI adapters +device isci # Intel C600 SAS controller + +# ATA/SCSI peripherals +device scbus # SCSI bus (required for ATA/SCSI) +device ch # SCSI media changers +device da # Direct Access (disks) +device sa # Sequential Access (tape etc) +device cd # CD +device pass # Passthrough device (direct ATA/SCSI access) +device ses # Enclosure Services (SES and SAF-TE) +#device ctl # CAM Target Layer + +# RAID controllers interfaced to the SCSI subsystem +device amr # AMI MegaRAID +device arcmsr # Areca SATA II RAID +device ciss # Compaq Smart RAID 5* +device dpt # DPT Smartcache III, IV - See NOTES for options +device hptmv # Highpoint RocketRAID 182x +device hptnr # Highpoint DC7280, R750 +device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx +device hpt27xx # Highpoint RocketRAID 27xx +device iir # Intel Integrated RAID +device ips # IBM (Adaptec) ServeRAID +device mly # Mylex AcceleRAID/eXtremeRAID +device twa # 3ware 9000 series PATA/SATA RAID +device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller + +# RAID controllers +device aac # Adaptec FSA RAID +device aacp # SCSI passthrough for aac (requires CAM) +device aacraid # Adaptec by PMC RAID +device ida # Compaq Smart RAID +device mfi # LSI MegaRAID SAS +device mlx # Mylex DAC960 family +device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s +#XXX pointer/int warnings +#device pst # Promise Supertrak SX6000 +device twe # 3ware ATA RAID + +# NVM Express (NVMe) support +device nvme # base NVMe driver +device nvd # expose NVMe namespaces as disks, depends on nvme + +# atkbdc0 controls both the keyboard and the PS/2 mouse +device atkbdc # AT keyboard controller +device atkbd # AT keyboard +device psm # PS/2 mouse + +device kbdmux # keyboard multiplexer + +device vga # VGA video card driver +options VESA # Add support for VESA BIOS Extensions (VBE) + +device splash # Splash screen and screen saver support + +# syscons is the default console driver, resembling an SCO console +device sc +options SC_PIXEL_MODE # add support for the raster text mode + +# vt is the new video console driver +device vt +device vt_vga +device vt_efifb + +device agp # support several AGP chipsets + +# PCCARD (PCMCIA) support +# PCMCIA and cardbus bridge support +device cbb # cardbus (yenta) bridge +device pccard # PC Card (16-bit) bus +device cardbus # CardBus (32-bit) bus + +# Serial (COM) ports +device uart # Generic UART driver + +# Parallel port +device ppc +device ppbus # Parallel port bus (required) +device lpt # Printer +device ppi # Parallel port interface device +#device vpo # Requires scbus and da + +device puc # Multi I/O cards and multi-channel UARTs + +# PCI Ethernet NICs. +device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE +device de # DEC/Intel DC21x4x (``Tulip'') +device em # Intel PRO/1000 Gigabit Ethernet Family +device igb # Intel PRO/1000 PCIE Server Gigabit Family +device ix # Intel PRO/10GbE PCIE PF Ethernet +device ixv # Intel PRO/10GbE PCIE VF Ethernet +device ixl # Intel XL710 40Gbe PCIE Ethernet +device ixlv # Intel XL710 40Gbe VF PCIE Ethernet +device le # AMD Am7900 LANCE and Am79C9xx PCnet +device ti # Alteon Networks Tigon I/II gigabit Ethernet +device txp # 3Com 3cR990 (``Typhoon'') +device vx # 3Com 3c590, 3c595 (``Vortex'') + +# PCI Ethernet NICs that use the common MII bus controller code. +# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! +device miibus # MII bus support +device ae # Attansic/Atheros L2 FastEthernet +device age # Attansic/Atheros L1 Gigabit Ethernet +device alc # Atheros AR8131/AR8132 Ethernet +device ale # Atheros AR8121/AR8113/AR8114 Ethernet +device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet +device bfe # Broadcom BCM440x 10/100 Ethernet +device bge # Broadcom BCM570xx Gigabit Ethernet +device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn +device dc # DEC/Intel 21143 and various workalikes +device et # Agere ET1310 10/100/Gigabit Ethernet +device fxp # Intel EtherExpress PRO/100B (82557, 82558) +device gem # Sun GEM/Sun ERI/Apple GMAC +device hme # Sun HME (Happy Meal Ethernet) +device jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet +device lge # Level 1 LXT1001 gigabit Ethernet +device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet +device nfe # nVidia nForce MCP on-board Ethernet +device nge # NatSemi DP83820 gigabit Ethernet +device pcn # AMD Am79C97x PCI 10/100 (precedence over 'le') +device re # RealTek 8139C+/8169/8169S/8110S +device rl # RealTek 8129/8139 +device sf # Adaptec AIC-6915 (``Starfire'') +device sge # Silicon Integrated Systems SiS190/191 +device sis # Silicon Integrated Systems SiS 900/SiS 7016 +device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet +device ste # Sundance ST201 (D-Link DFE-550TX) +device stge # Sundance/Tamarack TC9021 gigabit Ethernet +device tl # Texas Instruments ThunderLAN +device tx # SMC EtherPower II (83c170 ``EPIC'') +device vge # VIA VT612x gigabit Ethernet +device vr # VIA Rhine, Rhine II +device wb # Winbond W89C840F +device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') + +# Wireless NIC cards +device wlan # 802.11 support +options IEEE80211_DEBUG # enable debug msgs +options IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's +options IEEE80211_SUPPORT_MESH # enable 802.11s draft support +device wlan_wep # 802.11 WEP support +device wlan_ccmp # 802.11 CCMP support +device wlan_tkip # 802.11 TKIP support +device wlan_amrr # AMRR transmit rate control algorithm +device an # Aironet 4500/4800 802.11 wireless NICs. +device ath # Atheros NICs +device ath_pci # Atheros pci/cardbus glue +device ath_hal # pci/cardbus chip support +options AH_SUPPORT_AR5416 # enable AR5416 tx/rx descriptors +options AH_AR5416_INTERRUPT_MITIGATION # AR5416 interrupt mitigation +options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later +device ath_rate_sample # SampleRate tx rate control for ath +#device bwi # Broadcom BCM430x/BCM431x wireless NICs. +#device bwn # Broadcom BCM43xx wireless NICs. +device ipw # Intel 2100 wireless NICs. +device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. +device iwn # Intel 4965/1000/5000/6000 wireless NICs. +device malo # Marvell Libertas wireless NICs. +device mwl # Marvell 88W8363 802.11n wireless NICs. +device ral # Ralink Technology RT2500 wireless NICs. +device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. +device wpi # Intel 3945ABG wireless NICs. + +# Pseudo devices. +device loop # Network loopback +device random # Entropy device +device padlock_rng # VIA Padlock RNG +device rdrand_rng # Intel Bull Mountain RNG +device ether # Ethernet support +device vlan # 802.1Q VLAN support +device tun # Packet tunnel. +device md # Memory "disks" +device gif # IPv6 and IPv4 tunneling +device firmware # firmware assist module + +# The `bpf' device enables the Berkeley Packet Filter. +# Be aware of the administrative consequences of enabling this! +# Note that 'bpf' is required for DHCP. +device bpf # Berkeley packet filter + +# USB support +options USB_DEBUG # enable debug msgs +device uhci # UHCI PCI->USB interface +device ohci # OHCI PCI->USB interface +device ehci # EHCI PCI->USB interface (USB 2.0) +device xhci # XHCI PCI->USB interface (USB 3.0) +device usb # USB Bus (required) +device ukbd # Keyboard +device umass # Disks/Mass storage - Requires scbus and da + +# Sound support +device sound # Generic sound driver (required) +device snd_cmi # CMedia CMI8338/CMI8738 +device snd_csa # Crystal Semiconductor CS461x/428x +device snd_emu10kx # Creative SoundBlaster Live! and Audigy +device snd_es137x # Ensoniq AudioPCI ES137x +device snd_hda # Intel High Definition Audio +device snd_ich # Intel, NVidia and other ICH AC'97 Audio +device snd_via8233 # VIA VT8233x Audio + +# MMC/SD +device mmc # MMC/SD bus +device mmcsd # MMC/SD memory card +device sdhci # Generic PCI SD Host Controller + +# VirtIO support +device virtio # Generic VirtIO bus (required) +device virtio_pci # VirtIO PCI device +device vtnet # VirtIO Ethernet device +device virtio_blk # VirtIO Block device +device virtio_scsi # VirtIO SCSI device +device virtio_balloon # VirtIO Memory Balloon device + +# HyperV drivers and enchancement support +# NOTE: HYPERV depends on hyperv. They must be added or removed together. +options HYPERV # Hyper-V kernel infrastructure +device hyperv # HyperV drivers + +# Xen HVM Guest Optimizations +# NOTE: XENHVM depends on xenpci. They must be added or removed together. +options XENHVM # Xen HVM kernel infrastructure +device xenpci # Xen HVM Hypervisor services driver + +# VMware support +device vmx # VMware VMXNET3 Ethernet + +# Netmap provides direct access to TX/RX rings on supported NICs +device netmap # netmap(4) support + -- 2.32.0 From be8df23a906a9fccf35bbeaade9d29c9b4c703ac Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2015 17:30:20 +0200 Subject: [PATCH 014/325] ELECTRO_BLOAT: ElectroBSDify the copy Mainly by removing stuff that depends on proprietary software ElectroBSD doesn't ship with or stuff that isn't relevant for ElectroBSD. While at it, import the 'device crypto' line from GENERIC. While we don't need it for IPSEC, having it in the kernel makes upgrading from FreeBSD to ElectroBSD more convenient when using cloudiatr. By default cloudiatr only adds currenty-loaded modules to the bpool, so if crypto.ko is part of the currently running kernel, but the installed kernel requires it as module, the newly-setup system will not boot unless the user adds the module manually. Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 76 +++++++++++------------------------- 1 file changed, 22 insertions(+), 54 deletions(-) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index 3d2cc39c6353..a61f4e5eee3a 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -1,25 +1,25 @@ -# -# GENERIC -- Generic kernel configuration file for FreeBSD/amd64 -# -# For more information on this file, please read the config(5) manual page, -# and/or the handbook section on Kernel Configuration Files: -# -# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html -# -# The handbook is also available locally in /usr/share/doc/handbook -# if you've installed the doc distribution, otherwise always see the -# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the -# latest information. -# -# An exhaustive list of options and more detailed explanations of the -# device lines is also present in the ../../conf/NOTES and NOTES files. -# If you are in doubt as to the purpose or necessity of a line, check first -# in NOTES. -# -# $FreeBSD$ +# ELECTRO_BLOAT -- Modified copy of the GENERIC kernel configuration file +# Used for the release media. +ident ELECTRO_BLOAT + +# One of ElectroBSD's most important features: +# hacker-movie-compatible colors by default! +options SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK) + +# Add HTTP accept filter support. The "performance gains" might +# be dubious, but adding it results in nicer logs for applications +# that use it (because requests are less intangled). +options ACCEPT_FILTER_HTTP + +# Build pf into the kernel. It doesn't hurt and supposedly +# works around various bugs that only affect the module build. +device pf + +############################################################################## +# Everything below comes from GENERIC, but "offending" lines have been removed +############################################################################## cpu HAMMER -ident GENERIC makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support @@ -29,7 +29,6 @@ options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols options TCP_OFFLOAD # TCP offload -options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists @@ -84,10 +83,6 @@ options KDB_TRACE # Print a stack trace for a panic. options DDB # Support DDB. options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver -options INVARIANTS # Enable calls of extra sanity checking -options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS -options WITNESS # Enable checks to detect deadlocks and cycles -options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones # Make an SMP-capable kernel by default @@ -102,9 +97,6 @@ options ACPI_DMAR device pci options PCI_IOV # PCI SR-IOV support -# Floppy drives -device fdc - # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers @@ -121,13 +113,9 @@ options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series -device isp # Qlogic family -#device ispfw # Firmware for QLogic HBAs- normally a module device mpt # LSI-Logic MPT-Fusion device mps # LSI-Logic MPT-Fusion 2 device mpr # LSI-Logic MPT-Fusion 3 -#device ncr # NCR/Symbios Logic -device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') device trm # Tekram DC395U/UW/F DC315U adapters device adv # Advansys SCSI adapters @@ -151,10 +139,6 @@ device amr # AMI MegaRAID device arcmsr # Areca SATA II RAID device ciss # Compaq Smart RAID 5* device dpt # DPT Smartcache III, IV - See NOTES for options -device hptmv # Highpoint RocketRAID 182x -device hptnr # Highpoint DC7280, R750 -device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx -device hpt27xx # Highpoint RocketRAID 27xx device iir # Intel Integrated RAID device ips # IBM (Adaptec) ServeRAID device mly # Mylex AcceleRAID/eXtremeRAID @@ -167,10 +151,7 @@ device aacp # SCSI passthrough for aac (requires CAM) device aacraid # Adaptec by PMC RAID device ida # Compaq Smart RAID device mfi # LSI MegaRAID SAS -device mlx # Mylex DAC960 family device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s -#XXX pointer/int warnings -#device pst # Promise Supertrak SX6000 device twe # 3ware ATA RAID # NVM Express (NVMe) support @@ -219,7 +200,6 @@ device ppi # Parallel port interface device device puc # Multi I/O cards and multi-channel UARTs # PCI Ethernet NICs. -device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel PRO/1000 Gigabit Ethernet Family device igb # Intel PRO/1000 PCIE Server Gigabit Family @@ -239,7 +219,6 @@ device ae # Attansic/Atheros L2 FastEthernet device age # Attansic/Atheros L1 Gigabit Ethernet device alc # Atheros AR8131/AR8132 Ethernet device ale # Atheros AR8121/AR8113/AR8114 Ethernet -device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet device bfe # Broadcom BCM440x 10/100 Ethernet device bge # Broadcom BCM570xx Gigabit Ethernet device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn @@ -288,14 +267,7 @@ options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later device ath_rate_sample # SampleRate tx rate control for ath #device bwi # Broadcom BCM430x/BCM431x wireless NICs. #device bwn # Broadcom BCM43xx wireless NICs. -device ipw # Intel 2100 wireless NICs. -device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. -device iwn # Intel 4965/1000/5000/6000 wireless NICs. device malo # Marvell Libertas wireless NICs. -device mwl # Marvell 88W8363 802.11n wireless NICs. -device ral # Ralink Technology RT2500 wireless NICs. -device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. -device wpi # Intel 3945ABG wireless NICs. # Pseudo devices. device loop # Network loopback @@ -327,7 +299,6 @@ device umass # Disks/Mass storage - Requires scbus and da # Sound support device sound # Generic sound driver (required) device snd_cmi # CMedia CMI8338/CMI8738 -device snd_csa # Crystal Semiconductor CS461x/428x device snd_emu10kx # Creative SoundBlaster Live! and Audigy device snd_es137x # Ensoniq AudioPCI ES137x device snd_hda # Intel High Definition Audio @@ -348,8 +319,6 @@ device virtio_scsi # VirtIO SCSI device device virtio_balloon # VirtIO Memory Balloon device # HyperV drivers and enchancement support -# NOTE: HYPERV depends on hyperv. They must be added or removed together. -options HYPERV # Hyper-V kernel infrastructure device hyperv # HyperV drivers # Xen HVM Guest Optimizations @@ -360,6 +329,5 @@ device xenpci # Xen HVM Hypervisor services driver # VMware support device vmx # VMware VMXNET3 Ethernet -# Netmap provides direct access to TX/RX rings on supported NICs -device netmap # netmap(4) support - +# The crypto framework is required by IPSEC +device crypto # Required by IPSEC -- 2.32.0 From b6b4526c3aadf67f564fae4ad5d1d23fc0fd4bcd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 27 Mar 2013 11:28:27 +0100 Subject: [PATCH 015/325] Import ELECTRO_BEER Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BEER | 257 ++++++++++++++++++++++++++++++++++++ 1 file changed, 257 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BEER diff --git a/sys/amd64/conf/ELECTRO_BEER b/sys/amd64/conf/ELECTRO_BEER new file mode 100644 index 000000000000..3fb26ab1675f --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BEER @@ -0,0 +1,257 @@ +include ELECTRO_BLOAT + +ident ELECTRO_BEER + +# Debugging for use in -current +nooptions INVARIANTS # Enable calls of extra sanity checking +nooptions INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +nooptions WITNESS # Enable checks to detect deadlocks and cycles +nooptions WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +nooptions FLOWTABLE # per-cpu routing cache +nooptions XENHVM # Include Xen support + +nooptions SCTP # Stream Control Transmission Protocol + +nodevice fdc + +nodevice ataraid # ATA RAID drives +nodevice atapifd # ATAPI floppy drives +nodevice atapist # ATAPI tape drives + +# ATA controllers +nodevice mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA +nodevice siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA + +# SCSI Controllers +nodevice ahc # AHA2940 and onboard AIC7xxx devices +nooptions AHC_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~128k to driver. +nodevice amd # AMD 53C974 (Tekram DC-390(T)) +nodevice esp # AMD Am53C974 (Tekram DC-390(T)) +nodevice hptiop # Highpoint RocketRaid 3xxx series +nodevice isp # Qlogic family +nodevice ispfw # Firmware for QLogic HBAs- normally a module +nodevice mpt # LSI-Logic MPT-Fusion +nodevice mps # LSI-Logic MPT-Fusion 2 +nodevice ncr # NCR/Symbios Logic +nodevice sym # NCR/Symbios Logic (newer chipsets + those of `ncr') +nodevice trm # Tekram DC395U/UW/F DC315U adapters + +nodevice adv # Advansys SCSI adapters +nodevice adw # Advansys wide SCSI adapters +nodevice aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. +nodevice bt # Buslogic/Mylex MultiMaster SCSI adapters + +# RAID controllers interfaced to the SCSI subsystem +nodevice amr # AMI MegaRAID +nodevice arcmsr # Areca SATA II RAID +#XXX it is not 64-bit clean, -scottl +nodevice asr # DPT SmartRAID V, VI and Adaptec SCSI RAID +nodevice ciss # Compaq Smart RAID 5* +nodevice dpt # DPT Smartcache III, IV - See NOTES for options +nodevice hptmv # Highpoint RocketRAID 182x +nodevice hptnr # Highpoint DC7280, R750 +nodevice hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx +nodevice hpt27xx # Highpoint RocketRAID 27xx + +nodevice iir # Intel Integrated RAID +nodevice ips # IBM (Adaptec) ServeRAID +nodevice mly # Mylex AcceleRAID/eXtremeRAID +nodevice twa # 3ware 9000 series PATA/SATA RAID +nodevice tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller + +# RAID controllers +nodevice aac # Adaptec FSA RAID +nodevice aacraid # Adaptec by PMC RAID +nodevice aacp # SCSI passthrough for aac (requires CAM) +nodevice ida # Compaq Smart RAID +nodevice mfi # LSI MegaRAID SAS +nodevice mlx # Mylex DAC960 family +#XXX pointer/int warnings +nodevice pst # Promise Supertrak SX6000 +nodevice twe # 3ware ATA RAID + + +# Parallel port +nodevice ppc +nodevice ppbus # Parallel port bus (required) +nodevice lpt # Printer +nodevice plip # TCP/IP over parallel +nodevice ppi # Parallel port interface device +nodevice vpo # Requires scbus and da + +# If you've got a "dumb" serial or parallel PCI card that is +# supported by the puc(4) glue driver, uncomment the following +# line to enable it (connects to sio, uart and/or ppc drivers): +nodevice puc + +# PCI Ethernet NICs. +nodevice bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE +nodevice de # DEC/Intel DC21x4x (``Tulip'') +nodevice em # Intel PRO/1000 Gigabit Ethernet Family +nodevice igb # Intel PRO/1000 PCIE Server Gigabit Family +nodevice ixgbe # Intel PRO/10GbE PCIE Ethernet Family +nodevice le # AMD Am7900 LANCE and Am79C9xx PCnet +nodevice ti # Alteon Networks Tigon I/II gigabit Ethernet +nodevice txp # 3Com 3cR990 (``Typhoon'') +nodevice vx # 3Com 3c590, 3c595 (``Vortex'') + +# PCI Ethernet NICs that use the common MII bus controller code. +# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! + +nodevice ae # Attansic/Atheros L2 FastEthernet +nodevice age # Attansic/Atheros L1 Gigabit Ethernet +nodevice alc # Atheros AR8131/AR8132 Ethernet +nodevice ale # Atheros AR8121/AR8113/AR8114 Ethernet +nodevice bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet +nodevice bfe # Broadcom BCM440x 10/100 Ethernet +nodevice cas # Sun Cassini/Cassini+ and NS DP83065 Saturn +nodevice dc # DEC/Intel 21143 and various workalikes +nodevice et # Agere ET1310 10/100/Gigabit Ethernet +nodevice fxp # Intel EtherExpress PRO/100B (82557, 82558) +nodevice gem # Sun GEM/Sun ERI/Apple GMAC +nodevice hme # Sun HME (Happy Meal Ethernet) +nodevice jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet +nodevice lge # Level 1 LXT1001 gigabit Ethernet +nodevice msk # Marvell/SysKonnect Yukon II Gigabit Ethernet +nodevice nfe # nVidia nForce MCP on-board Ethernet +nodevice nge # NatSemi DP83820 gigabit Ethernet +nodevice nve # nVidia nForce MCP on-board Ethernet Networking +nodevice pcn # AMD Am79C97x PCI 10/100 (precedence over 'le') +nodevice re # RealTek 8139C+/8169/8169S/8110S +nodevice rl # RealTek 8129/8139 +nodevice sf # Adaptec AIC-6915 (``Starfire'') +nodevice sge # Silicon Integrated Systems SiS190/191 +nodevice sis # Silicon Integrated Systems SiS 900/SiS 7016 +nodevice sk # SysKonnect SK-984x & SK-982x gigabit Ethernet +nodevice ste # Sundance ST201 (D-Link DFE-550TX) +nodevice stge # Sundance/Tamarack TC9021 gigabit Ethernet +nodevice tl # Texas Instruments ThunderLAN +nodevice tx # SMC EtherPower II (83c170 ``EPIC'') +nodevice vge # VIA VT612x gigabit Ethernet +nodevice vr # VIA Rhine, Rhine II +nodevice wb # Winbond W89C840F +nodevice xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') + +# ISA Ethernet NICs. pccard NICs included. +nodevice cs # Crystal Semiconductor CS89x0 NIC +# 'device ed' requires 'device miibus' +nodevice ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards +nodevice ex # Intel EtherExpress Pro/10 and Pro/10+ +nodevice ep # Etherlink III based cards +nodevice fe # Fujitsu MB8696x based cards +nodevice sn # SMC's 9000 series of Ethernet chips +nodevice xe # Xircom pccard Ethernet + +nodevice zyd # ZyDAS zb1211/zb1211b wireless NICs +nodevice urio # Diamond Rio 500 MP3 player + +# Wireless NIC cards +nodevice an # Aironet 4500/4800 802.11 wireless NICs. +nodevice ath # Atheros NIC's +nodevice ath_pci # Atheros pci/cardbus glue +nodevice ath_hal # pci/cardbus chip support +nodevice ath_rate_sample # SampleRate tx rate control for ath +nodevice bwi # Broadcom BCM430x/BCM431x wireless NICs. +nodevice bwn # Broadcom BCM43xx wireless NICs. +nodevice ipw # Intel 2100 wireless NICs. +nodevice iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. +nodevice iwn # Intel 4965/1000/5000/6000 wireless NICs. +nodevice malo # Marvell Libertas wireless NICs. +nodevice mwl # Marvell 88W8363 802.11n wireless NICs. +nodevice ral # Ralink Technology RT2500 wireless NICs. +nodevice wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. +nodevice wpi # Intel 3945ABG wireless NICs. + +# Pseudo devices. +nodevice padlock_rng # VIA Padlock RNG +nodevice rdrand_rng # Intel Bull Mountain RNG +nodevice gif # IPv6 and IPv4 tunneling +nodevice faith # IPv6-to-IPv4 relaying (translation) + +# USB Serial devices +nodevice uark # Technologies ARK3116 based serial adapters +nodevice ubsa # Belkin F5U103 and compatible serial adapters +nodevice uftdi # For FTDI usb serial adapters +nodevice uipaq # Some WinCE based devices +nodevice uplcom # Prolific PL-2303 serial adapters +nodevice uslcom # SI Labs CP2101/CP2102 serial adapters +nodevice uvisor # Visor and Palm devices +nodevice uvscom # USB serial support for DDI pocket's PHS +# USB Ethernet, requires miibus +nodevice aue # ADMtek USB Ethernet +nodevice axe # ASIX Electronics USB Ethernet +nodevice cdce # Generic USB over Ethernet +nodevice cue # CATC USB Ethernet +nodevice kue # Kawasaki LSI USB Ethernet +nodevice rue # RealTek RTL8150 USB Ethernet +nodevice udav # Davicom DM9601E USB + +# USB support + +nodevice uhci # UHCI PCI->USB interface +nodevice ohci # OHCI PCI->USB interface +nodevice ehci # EHCI PCI->USB interface (USB 2.0) +nodevice xhci # XHCI PCI->USB interface (USB 3.0) +nodevice usb # USB Bus (required) +nodevice ukbd # Keyboard +nodevice umass # Disks/Mass storage - Requires scbus and da + +# FireWire support +nodevice firewire # FireWire bus code +nodevice sbp # SCSI over FireWire (Requires scbus and da) +nodevice fwe # Ethernet over FireWire (non-standard!) +nodevice fwip # IP over FireWire (RFC 2734,3146) +nodevice dcons # Dumb console driver +nodevice dcons_crom # Configuration ROM for dcons + +# Sound support +nodevice snd_es137x # Ensoniq AudioPCI ES137x +nodevice snd_ich # Intel, NVidia and other ICH AC'97 Audio +nodevice snd_uaudio # USB Audio +nodevice snd_via8233 # VIA VT8233x Audio +nodevice snd_cmi # CMedia CMI8338/CMI8738 +nodevice snd_csa # Crystal Semiconductor CS461x/428x +nodevice snd_emu10kx # Creative SoundBlaster Live! and Audigy + +# VirtIO support +nodevice virtio # Generic VirtIO bus (required) +nodevice virtio_pci # VirtIO PCI device +nodevice vtnet # VirtIO Ethernet device +nodevice virtio_blk # VirtIO Block device +nodevice virtio_scsi # VirtIO SCSI device +nodevice virtio_balloon # VirtIO Memory Balloon device + +# HyperV drivers +nodevice hyperv # HyperV drivers + +# Xen support +nodevice xenpci # Generic Xen bus + +# VMware support +nodevice vmx # VMware VMXNET3 Ethernet + +# Same for Intel processors +device coretemp + +# man 4 cpuctl +device cpuctl # CPU control pseudo-device + +# UTF-8 in console (8.x+) +options TEKEN_UTF8 + +nodevice netmap + +# Disabling them prevents ZFS from being loaded. +# Should be bisected. +#nooptions NFS_ROOT +#nooptions NFSLOCKD +#nooptions NFSD +#nooptions NFSCL +nodevice ahd +nodevice mpr +nodevice isci +nodevice ses +nodevice mrsas +nodevice ixl +nodevice ixlv -- 2.32.0 From 8bc4b002861468c800ab4931b293cb1489f608bb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 14 Apr 2015 17:43:38 +0200 Subject: [PATCH 016/325] Add ELECTRO_BEER for i386 Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BEER | 1 + 1 file changed, 1 insertion(+) create mode 120000 sys/i386/conf/ELECTRO_BEER diff --git a/sys/i386/conf/ELECTRO_BEER b/sys/i386/conf/ELECTRO_BEER new file mode 120000 index 000000000000..4a483bb3a8ea --- /dev/null +++ b/sys/i386/conf/ELECTRO_BEER @@ -0,0 +1 @@ +../../amd64/conf/ELECTRO_BEER \ No newline at end of file -- 2.32.0 From e86db875d01ad781fd07c4b9dc4638375a585bfb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 28 Jun 2015 17:06:35 +0200 Subject: [PATCH 017/325] i386: Copy GENERIC to ELECTRO_BLOAT Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BLOAT | 382 ++++++++++++++++++++++++++++++++++++ 1 file changed, 382 insertions(+) create mode 100644 sys/i386/conf/ELECTRO_BLOAT diff --git a/sys/i386/conf/ELECTRO_BLOAT b/sys/i386/conf/ELECTRO_BLOAT new file mode 100644 index 000000000000..aefc50753f0d --- /dev/null +++ b/sys/i386/conf/ELECTRO_BLOAT @@ -0,0 +1,382 @@ +# +# GENERIC -- Generic kernel configuration file for FreeBSD/i386 +# +# For more information on this file, please read the config(5) manual page, +# and/or the handbook section on Kernel Configuration Files: +# +# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html +# +# The handbook is also available locally in /usr/share/doc/handbook +# if you've installed the doc distribution, otherwise always see the +# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the +# latest information. +# +# An exhaustive list of options and more detailed explanations of the +# device lines is also present in the ../../conf/NOTES and NOTES files. +# If you are in doubt as to the purpose or necessity of a line, check first +# in NOTES. +# +# $FreeBSD$ + +cpu I486_CPU +cpu I586_CPU +cpu I686_CPU +ident GENERIC + +makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols +makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support + +options SCHED_ULE # ULE scheduler +options PREEMPTION # Enable kernel thread preemption +options INET # InterNETworking +options INET6 # IPv6 communications protocols +options IPSEC # IP (v4/v6) security +options TCP_OFFLOAD # TCP offload +options SCTP # Stream Control Transmission Protocol +options FFS # Berkeley Fast Filesystem +options SOFTUPDATES # Enable FFS soft updates support +options UFS_ACL # Support for access control lists +options UFS_DIRHASH # Improve performance on big directories +options UFS_GJOURNAL # Enable gjournal-based UFS journaling +options QUOTA # Enable disk quotas for UFS +options MD_ROOT # MD is a potential root device +options NFSCL # Network Filesystem Client +options NFSD # Network Filesystem Server +options NFSLOCKD # Network Lock Manager +options NFS_ROOT # NFS usable as /, requires NFSCL +options MSDOSFS # MSDOS Filesystem +options CD9660 # ISO 9660 Filesystem +options PROCFS # Process filesystem (requires PSEUDOFS) +options PSEUDOFS # Pseudo-filesystem framework +options GEOM_PART_GPT # GUID Partition Tables. +options GEOM_RAID # Soft RAID functionality. +options GEOM_LABEL # Provides labelization +options COMPAT_FREEBSD4 # Compatible with FreeBSD4 +options COMPAT_FREEBSD5 # Compatible with FreeBSD5 +options COMPAT_FREEBSD6 # Compatible with FreeBSD6 +options COMPAT_FREEBSD7 # Compatible with FreeBSD7 +options COMPAT_FREEBSD9 # Compatible with FreeBSD9 +options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI +options KTRACE # ktrace(1) support +options STACK # stack(9) support +options SYSVSHM # SYSV-style shared memory +options SYSVMSG # SYSV-style message queues +options SYSVSEM # SYSV-style semaphores +options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions +options PRINTF_BUFR_SIZE=128 # Prevent printf output being interspersed. +options KBD_INSTALL_CDEV # install a CDEV entry in /dev +options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(4) +options AUDIT # Security event auditing +options CAPABILITY_MODE # Capsicum capability mode +options CAPABILITIES # Capsicum capabilities +options MAC # TrustedBSD MAC Framework +options KDTRACE_HOOKS # Kernel DTrace hooks +options DDB_CTF # Kernel ELF linker loads CTF data +options INCLUDE_CONFIG_FILE # Include this file in kernel +options RACCT # Resource accounting framework +options RACCT_DEFAULT_TO_DISABLED # Set kern.racct.enable=0 by default +options RCTL # Resource limits + +# Debugging support. Always need this: +options KDB # Enable kernel debugger support. +options KDB_TRACE # Print a stack trace for a panic. +# For full debugger support use (turn off in stable branch): +options DDB # Support DDB. +options GDB # Support remote GDB. +options DEADLKRES # Enable the deadlock resolver +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones + +# To make an SMP kernel, the next two lines are needed +options SMP # Symmetric MultiProcessor Kernel +device apic # I/O APIC + +# CPU frequency control +device cpufreq + +# Bus support. +device acpi +device pci +options PCI_HP # PCI-Express native HotPlug +options PCI_IOV # PCI SR-IOV support + +# Floppy drives +device fdc + +# ATA controllers +device ahci # AHCI-compatible SATA controllers +device ata # Legacy ATA/SATA controllers +device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA +device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA + +# SCSI Controllers +device ahc # AHA2940 and onboard AIC7xxx devices +options AHC_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~128k to driver. +device ahd # AHA39320/29320 and onboard AIC79xx devices +options AHD_REG_PRETTY_PRINT # Print register bitfields in debug + # output. Adds ~215k to driver. +device esp # AMD Am53C974 (Tekram DC-390(T)) +device hptiop # Highpoint RocketRaid 3xxx series +device isp # Qlogic family +#device ispfw # Firmware for QLogic HBAs- normally a module +device mpt # LSI-Logic MPT-Fusion +device mps # LSI-Logic MPT-Fusion 2 +device mpr # LSI-Logic MPT-Fusion 3 +#device ncr # NCR/Symbios Logic +device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') +device trm # Tekram DC395U/UW/F DC315U adapters + +device adv # Advansys SCSI adapters +device adw # Advansys wide SCSI adapters +device aha # Adaptec 154x SCSI adapters +device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. +device bt # Buslogic/Mylex MultiMaster SCSI adapters + +device ncv # NCR 53C500 +device nsp # Workbit Ninja SCSI-3 +device stg # TMC 18C30/18C50 +device isci # Intel C600 SAS controller + +# ATA/SCSI peripherals +device scbus # SCSI bus (required for ATA/SCSI) +device ch # SCSI media changers +device da # Direct Access (disks) +device sa # Sequential Access (tape etc) +device cd # CD +device pass # Passthrough device (direct ATA/SCSI access) +device ses # Enclosure Services (SES and SAF-TE) +#device ctl # CAM Target Layer + +# RAID controllers interfaced to the SCSI subsystem +device amr # AMI MegaRAID +device arcmsr # Areca SATA II RAID +device ciss # Compaq Smart RAID 5* +device dpt # DPT Smartcache III, IV - See NOTES for options +device hptmv # Highpoint RocketRAID 182x +device hptnr # Highpoint DC7280, R750 +device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx +device hpt27xx # Highpoint RocketRAID 27xx +device iir # Intel Integrated RAID +device ips # IBM (Adaptec) ServeRAID +device mly # Mylex AcceleRAID/eXtremeRAID +device twa # 3ware 9000 series PATA/SATA RAID +device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller + +# RAID controllers +device aac # Adaptec FSA RAID +device aacp # SCSI passthrough for aac (requires CAM) +device aacraid # Adaptec by PMC RAID +device ida # Compaq Smart RAID +device mfi # LSI MegaRAID SAS +device mlx # Mylex DAC960 family +device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s +device pmspcv # PMC-Sierra SAS/SATA Controller driver +device pst # Promise Supertrak SX6000 +device twe # 3ware ATA RAID + +# NVM Express (NVMe) support +device nvme # base NVMe driver +device nvd # expose NVMe namespace as disks, depends on nvme + +# atkbdc0 controls both the keyboard and the PS/2 mouse +device atkbdc # AT keyboard controller +device atkbd # AT keyboard +device psm # PS/2 mouse + +device kbdmux # keyboard multiplexer + +device vga # VGA video card driver +options VESA # Add support for VESA BIOS Extensions (VBE) + +device splash # Splash screen and screen saver support + +# syscons is the default console driver, resembling an SCO console +device sc +options SC_PIXEL_MODE # add support for the raster text mode + +# vt is the new video console driver +device vt +device vt_vga + +device agp # support several AGP chipsets + +# Power management support (see NOTES for more options) +#device apm +# Add suspend/resume support for the i8254. +device pmtimer + +# PCCARD (PCMCIA) support +# PCMCIA and cardbus bridge support +device cbb # cardbus (yenta) bridge +device pccard # PC Card (16-bit) bus +device cardbus # CardBus (32-bit) bus + +# Serial (COM) ports +device uart # Generic UART driver + +# Parallel port +device ppc +device ppbus # Parallel port bus (required) +device lpt # Printer +device ppi # Parallel port interface device +#device vpo # Requires scbus and da + +device puc # Multi I/O cards and multi-channel UARTs + +# PCI Ethernet NICs. +device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE +device de # DEC/Intel DC21x4x (``Tulip'') +device em # Intel PRO/1000 Gigabit Ethernet Family +device igb # Intel PRO/1000 PCIE Server Gigabit Family +device ixgb # Intel PRO/10GbE Ethernet Card +device le # AMD Am7900 LANCE and Am79C9xx PCnet +device ti # Alteon Networks Tigon I/II gigabit Ethernet +device txp # 3Com 3cR990 (``Typhoon'') +device vx # 3Com 3c590, 3c595 (``Vortex'') + +# PCI Ethernet NICs that use the common MII bus controller code. +# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! +device miibus # MII bus support +device ae # Attansic/Atheros L2 FastEthernet +device age # Attansic/Atheros L1 Gigabit Ethernet +device alc # Atheros AR8131/AR8132 Ethernet +device ale # Atheros AR8121/AR8113/AR8114 Ethernet +device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet +device bfe # Broadcom BCM440x 10/100 Ethernet +device bge # Broadcom BCM570xx Gigabit Ethernet +device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn +device dc # DEC/Intel 21143 and various workalikes +device et # Agere ET1310 10/100/Gigabit Ethernet +device fxp # Intel EtherExpress PRO/100B (82557, 82558) +device gem # Sun GEM/Sun ERI/Apple GMAC +device hme # Sun HME (Happy Meal Ethernet) +device jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet +device lge # Level 1 LXT1001 gigabit Ethernet +device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet +device nfe # nVidia nForce MCP on-board Ethernet +device nge # NatSemi DP83820 gigabit Ethernet +device pcn # AMD Am79C97x PCI 10/100 (precedence over 'le') +device re # RealTek 8139C+/8169/8169S/8110S +device rl # RealTek 8129/8139 +device sf # Adaptec AIC-6915 (``Starfire'') +device sge # Silicon Integrated Systems SiS190/191 +device sis # Silicon Integrated Systems SiS 900/SiS 7016 +device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet +device ste # Sundance ST201 (D-Link DFE-550TX) +device stge # Sundance/Tamarack TC9021 gigabit Ethernet +device tl # Texas Instruments ThunderLAN +device tx # SMC EtherPower II (83c170 ``EPIC'') +device vge # VIA VT612x gigabit Ethernet +device vr # VIA Rhine, Rhine II +device vte # DM&P Vortex86 RDC R6040 Fast Ethernet +device wb # Winbond W89C840F +device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') + +# ISA Ethernet NICs. pccard NICs included. +device cs # Crystal Semiconductor CS89x0 NIC +# 'device ed' requires 'device miibus' +device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards +device ex # Intel EtherExpress Pro/10 and Pro/10+ +device ep # Etherlink III based cards +device fe # Fujitsu MB8696x based cards +device ie # EtherExpress 8/16, 3C507, StarLAN 10 etc. +device sn # SMC's 9000 series of Ethernet chips +device xe # Xircom pccard Ethernet + +# Wireless NIC cards +device wlan # 802.11 support +options IEEE80211_DEBUG # enable debug msgs +options IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's +options IEEE80211_SUPPORT_MESH # enable 802.11s draft support +device wlan_wep # 802.11 WEP support +device wlan_ccmp # 802.11 CCMP support +device wlan_tkip # 802.11 TKIP support +device wlan_amrr # AMRR transmit rate control algorithm +device an # Aironet 4500/4800 802.11 wireless NICs. +device ath # Atheros NICs +device ath_pci # Atheros pci/cardbus glue +device ath_hal # pci/cardbus chip support +options AH_SUPPORT_AR5416 # enable AR5416 tx/rx descriptors +options AH_AR5416_INTERRUPT_MITIGATION # AR5416 interrupt mitigation +options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later +device ath_rate_sample # SampleRate tx rate control for ath +#device bwi # Broadcom BCM430x/BCM431x wireless NICs. +#device bwn # Broadcom BCM43xx wireless NICs. +device ipw # Intel 2100 wireless NICs. +device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. +device iwn # Intel 4965/1000/5000/6000 wireless NICs. +device malo # Marvell Libertas wireless NICs. +device mwl # Marvell 88W8363 802.11n wireless NICs. +device ral # Ralink Technology RT2500 wireless NICs. +device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. +#device wl # Older non 802.11 Wavelan wireless NIC. +device wpi # Intel 3945ABG wireless NICs. + +# Pseudo devices. +device loop # Network loopback +device random # Entropy device +device padlock_rng # VIA Padlock RNG +device rdrand_rng # Intel Bull Mountain RNG +device ether # Ethernet support +device vlan # 802.1Q VLAN support +device tun # Packet tunnel. +device md # Memory "disks" +device gif # IPv6 and IPv4 tunneling +device firmware # firmware assist module + +# The `bpf' device enables the Berkeley Packet Filter. +# Be aware of the administrative consequences of enabling this! +# Note that 'bpf' is required for DHCP. +device bpf # Berkeley packet filter + +# USB support +options USB_DEBUG # enable debug msgs +device uhci # UHCI PCI->USB interface +device ohci # OHCI PCI->USB interface +device ehci # EHCI PCI->USB interface (USB 2.0) +device xhci # XHCI PCI->USB interface (USB 3.0) +device usb # USB Bus (required) +device ukbd # Keyboard +device umass # Disks/Mass storage - Requires scbus and da + +# Sound support +device sound # Generic sound driver (required) +device snd_cmi # CMedia CMI8338/CMI8738 +device snd_csa # Crystal Semiconductor CS461x/428x +device snd_emu10kx # Creative SoundBlaster Live! and Audigy +device snd_es137x # Ensoniq AudioPCI ES137x +device snd_hda # Intel High Definition Audio +device snd_ich # Intel, NVidia and other ICH AC'97 Audio +device snd_via8233 # VIA VT8233x Audio + +# MMC/SD +device mmc # MMC/SD bus +device mmcsd # MMC/SD memory card +device sdhci # Generic PCI SD Host Controller + +# VirtIO support +device virtio # Generic VirtIO bus (required) +device virtio_pci # VirtIO PCI device +device vtnet # VirtIO Ethernet device +device virtio_blk # VirtIO Block device +device virtio_scsi # VirtIO SCSI device +device virtio_balloon # VirtIO Memory Balloon device + +# HyperV drivers and enchancement support +device hyperv # HyperV drivers + +# Xen HVM Guest Optimizations +# NOTE: XENHVM depends on xenpci. They must be added or removed together. +options XENHVM # Xen HVM kernel infrastructure +device xenpci # Xen HVM Hypervisor services driver + +# VMware support +device vmx # VMware VMXNET3 Ethernet + +# The crypto framework is required by IPSEC +device crypto # Required by IPSEC -- 2.32.0 From 51f8b1384fa3b93cf718e280a791352d244062cc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 28 Jun 2015 17:55:42 +0200 Subject: [PATCH 018/325] Adjust shiny new ELECTRO_BLOAT i386 for ElectroBSD Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BLOAT | 81 ++++++++----------------------------- 1 file changed, 17 insertions(+), 64 deletions(-) diff --git a/sys/i386/conf/ELECTRO_BLOAT b/sys/i386/conf/ELECTRO_BLOAT index aefc50753f0d..fcfb7c314411 100644 --- a/sys/i386/conf/ELECTRO_BLOAT +++ b/sys/i386/conf/ELECTRO_BLOAT @@ -1,27 +1,11 @@ -# -# GENERIC -- Generic kernel configuration file for FreeBSD/i386 -# -# For more information on this file, please read the config(5) manual page, -# and/or the handbook section on Kernel Configuration Files: -# -# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html -# -# The handbook is also available locally in /usr/share/doc/handbook -# if you've installed the doc distribution, otherwise always see the -# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the -# latest information. -# -# An exhaustive list of options and more detailed explanations of the -# device lines is also present in the ../../conf/NOTES and NOTES files. -# If you are in doubt as to the purpose or necessity of a line, check first -# in NOTES. -# -# $FreeBSD$ - -cpu I486_CPU -cpu I586_CPU +# ELECTRO_BLOAT -- Modified copy of the GENERIC kernel configuration file +# Used for the release media. + cpu I686_CPU -ident GENERIC +ident ELECTRO_BLOAT + +# Hacker-movie-compatible colors +options SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK) makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support @@ -30,9 +14,7 @@ options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols -options IPSEC # IP (v4/v6) security options TCP_OFFLOAD # TCP offload -options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists @@ -49,12 +31,7 @@ options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. -options GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization -options COMPAT_FREEBSD4 # Compatible with FreeBSD4 -options COMPAT_FREEBSD5 # Compatible with FreeBSD5 -options COMPAT_FREEBSD6 # Compatible with FreeBSD6 -options COMPAT_FREEBSD7 # Compatible with FreeBSD7 options COMPAT_FREEBSD9 # Compatible with FreeBSD9 options COMPAT_FREEBSD10 # Compatible with FreeBSD10 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI @@ -85,10 +62,6 @@ options KDB_TRACE # Print a stack trace for a panic. options DDB # Support DDB. options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver -options INVARIANTS # Enable calls of extra sanity checking -options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS -options WITNESS # Enable checks to detect deadlocks and cycles -options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones # To make an SMP kernel, the next two lines are needed @@ -101,15 +74,12 @@ device cpufreq # Bus support. device acpi device pci -options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support -# Floppy drives -device fdc - # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers +options ATA_STATIC_ID # Static device numbering device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA @@ -123,12 +93,9 @@ options AHD_REG_PRETTY_PRINT # Print register bitfields in debug device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series device isp # Qlogic family -#device ispfw # Firmware for QLogic HBAs- normally a module device mpt # LSI-Logic MPT-Fusion device mps # LSI-Logic MPT-Fusion 2 device mpr # LSI-Logic MPT-Fusion 3 -#device ncr # NCR/Symbios Logic -device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') device trm # Tekram DC395U/UW/F DC315U adapters device adv # Advansys SCSI adapters @@ -157,10 +124,6 @@ device amr # AMI MegaRAID device arcmsr # Areca SATA II RAID device ciss # Compaq Smart RAID 5* device dpt # DPT Smartcache III, IV - See NOTES for options -device hptmv # Highpoint RocketRAID 182x -device hptnr # Highpoint DC7280, R750 -device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx -device hpt27xx # Highpoint RocketRAID 27xx device iir # Intel Integrated RAID device ips # IBM (Adaptec) ServeRAID device mly # Mylex AcceleRAID/eXtremeRAID @@ -173,16 +136,10 @@ device aacp # SCSI passthrough for aac (requires CAM) device aacraid # Adaptec by PMC RAID device ida # Compaq Smart RAID device mfi # LSI MegaRAID SAS -device mlx # Mylex DAC960 family device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s -device pmspcv # PMC-Sierra SAS/SATA Controller driver device pst # Promise Supertrak SX6000 device twe # 3ware ATA RAID -# NVM Express (NVMe) support -device nvme # base NVMe driver -device nvd # expose NVMe namespace as disks, depends on nvme - # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard @@ -205,8 +162,6 @@ device vt_vga device agp # support several AGP chipsets -# Power management support (see NOTES for more options) -#device apm # Add suspend/resume support for the i8254. device pmtimer @@ -229,7 +184,6 @@ device ppi # Parallel port interface device device puc # Multi I/O cards and multi-channel UARTs # PCI Ethernet NICs. -device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel PRO/1000 Gigabit Ethernet Family device igb # Intel PRO/1000 PCIE Server Gigabit Family @@ -246,7 +200,6 @@ device ae # Attansic/Atheros L2 FastEthernet device age # Attansic/Atheros L1 Gigabit Ethernet device alc # Atheros AR8131/AR8132 Ethernet device ale # Atheros AR8121/AR8113/AR8114 Ethernet -device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet device bfe # Broadcom BCM440x 10/100 Ethernet device bge # Broadcom BCM570xx Gigabit Ethernet device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn @@ -307,15 +260,7 @@ options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later device ath_rate_sample # SampleRate tx rate control for ath #device bwi # Broadcom BCM430x/BCM431x wireless NICs. #device bwn # Broadcom BCM43xx wireless NICs. -device ipw # Intel 2100 wireless NICs. -device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. -device iwn # Intel 4965/1000/5000/6000 wireless NICs. device malo # Marvell Libertas wireless NICs. -device mwl # Marvell 88W8363 802.11n wireless NICs. -device ral # Ralink Technology RT2500 wireless NICs. -device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. -#device wl # Older non 802.11 Wavelan wireless NIC. -device wpi # Intel 3945ABG wireless NICs. # Pseudo devices. device loop # Network loopback @@ -347,7 +292,6 @@ device umass # Disks/Mass storage - Requires scbus and da # Sound support device sound # Generic sound driver (required) device snd_cmi # CMedia CMI8338/CMI8738 -device snd_csa # Crystal Semiconductor CS461x/428x device snd_emu10kx # Creative SoundBlaster Live! and Audigy device snd_es137x # Ensoniq AudioPCI ES137x device snd_hda # Intel High Definition Audio @@ -380,3 +324,12 @@ device vmx # VMware VMXNET3 Ethernet # The crypto framework is required by IPSEC device crypto # Required by IPSEC + +# This used to be required for ZFS when compiled with clang. +# For details see UPDATING entry 20121223. After r286288 it's +# probably no longer necessary, but for now we keep it anyway. +options KSTACK_PAGES=4 + +# Increase the size of the kernel virtual address space +# so ZFS can cache more stuff. +options KVA_PAGES=512 -- 2.32.0 From 72e6b9439f5ff738babc6d7472e21a1902787685 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2015 17:38:09 +0200 Subject: [PATCH 019/325] Change amd64 default KERNCONF to ELECTRO_BLOAT Obtained from: ElectroBSD --- Makefile.inc1 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Makefile.inc1 b/Makefile.inc1 index 1b1f8b162d92..128efe3ca12c 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1600,6 +1600,8 @@ KERNCONF=${KERNFAST} .endif .if ${TARGET_ARCH} == "powerpc64" KERNCONF?= GENERIC64 +.elif ${TARGET_ARCH} == "amd64" +KERNCONF?= ELECTRO_BLOAT .else KERNCONF?= GENERIC .endif -- 2.32.0 From f46cf9156eaaf56ddbc4344a5a3ee2c889a5c122 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 22 May 2016 12:10:03 +0200 Subject: [PATCH 020/325] Use ELECTRO_BLOAT as default on i386 Obtained from: ElectroBSD --- Makefile.inc1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.inc1 b/Makefile.inc1 index 128efe3ca12c..1c9058342252 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1600,7 +1600,7 @@ KERNCONF=${KERNFAST} .endif .if ${TARGET_ARCH} == "powerpc64" KERNCONF?= GENERIC64 -.elif ${TARGET_ARCH} == "amd64" +.elif ${TARGET_ARCH} == "amd64" || ${TARGET_ARCH} == "i386" KERNCONF?= ELECTRO_BLOAT .else KERNCONF?= GENERIC -- 2.32.0 From 9866e8ba03db3696a79be8dbd9184f8bc147c168 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Aug 2011 19:19:51 +0200 Subject: [PATCH 021/325] Register the product id for Feiya Memory Bar Obtained from: ElectroBSD --- sys/dev/usb/usbdevs | 1 + 1 file changed, 1 insertion(+) diff --git a/sys/dev/usb/usbdevs b/sys/dev/usb/usbdevs index 15cd996745e6..8b1142e561a4 100644 --- a/sys/dev/usb/usbdevs +++ b/sys/dev/usb/usbdevs @@ -1972,6 +1972,7 @@ product FALCOM SAMBA 0x0005 FTDI compatible adapter /* FEIYA products */ product FEIYA DUMMY 0x0000 Dummy product +product FEIYA MEMORY_BAR 0x1000 Memory Bar product FEIYA 5IN1 0x1132 5-in-1 Card Reader product FEIYA ELANGO 0x6200 MicroSDHC Card Reader product FEIYA AC110 0x6300 AC-110 Card Reader -- 2.32.0 From 75b5c3d5f72ca3fed0435bd6696f32117e431efc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Aug 2011 19:22:46 +0200 Subject: [PATCH 022/325] Add the full name of 'Feya Technology Corp.' Obtained from: ElectroBSD --- sys/dev/usb/usbdevs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/usb/usbdevs b/sys/dev/usb/usbdevs index 8b1142e561a4..749b1e707e6b 100644 --- a/sys/dev/usb/usbdevs +++ b/sys/dev/usb/usbdevs @@ -455,7 +455,7 @@ vendor DIGIANSWER 0x08fd Digianswer vendor AUTHENTEC 0x08ff AuthenTec vendor AUDIOTECHNICA 0x0909 Audio-Technica vendor TRUMPION 0x090a Trumpion Microelectronics -vendor FEIYA 0x090c Feiya +vendor FEIYA 0x090c Feiya Technology Corp. vendor ALATION 0x0910 Alation Systems vendor GLOBESPAN 0x0915 Globespan vendor CONCORDCAMERA 0x0919 Concord Camera -- 2.32.0 From 8b30dc3f9123150756bef01e96031311cfa6c9d7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 20 Dec 2013 18:45:00 +0100 Subject: [PATCH 023/325] Let g_eli_keyfiles_load() log the size of the keyfile loaded Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index a763731b0c63..8411063478e2 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -1019,8 +1019,8 @@ g_eli_keyfiles_load(struct hmac_ctx *ctx, const char *provider) name); return (0); } - G_ELI_DEBUG(1, "Loaded keyfile %s for %s (type: %s).", file, - provider, name); + G_ELI_DEBUG(1, "Loaded keyfile %s for %s (type: %s) (size: %d).", file, + provider, name, (unsigned)size); g_eli_crypto_hmac_update(ctx, data, size); } } -- 2.32.0 From c057ab2a74f5262978d0d05d3f40ec2916d329fb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 7 Jan 2015 18:50:18 +0100 Subject: [PATCH 024/325] bge(4): Default to disallowing ASF It causes watchdog timeouts and undiagnosed permanent unresponsivenes on at least the 'CHIP ID 0x05784100; ASIC REV 0x5784; CHIP REV 0x57841;' in the evo-iv08 DL120 G6 I'm using for testing. I don't have time to debug the underlying cause right now and users who actually want ASF and have systems where it works can always enable it through loader.conf. Obtained from: ElectroBSD --- share/man/man4/bge.4 | 2 +- sys/dev/bge/if_bge.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/share/man/man4/bge.4 b/share/man/man4/bge.4 index 50c0cf9c9b8c..c09b35edc088 100644 --- a/share/man/man4/bge.4 +++ b/share/man/man4/bge.4 @@ -200,7 +200,7 @@ prompt before booting the kernel, or stored in .It Va hw.bge.allow_asf Allow the ASF feature for cooperating with IPMI. Can cause system lockup problems on a small number of systems. -Enabled by default. +Disabled by default. .It Va dev.bge.%d.msi Non-zero value enables MSI support on the Ethernet hardware. The default value is 1. diff --git a/sys/dev/bge/if_bge.c b/sys/dev/bge/if_bge.c index bae18acf96ae..9b4a5d04ea29 100644 --- a/sys/dev/bge/if_bge.c +++ b/sys/dev/bge/if_bge.c @@ -551,7 +551,7 @@ MODULE_PNP_INFO("U16:vendor;U16:device", pci, bge, bge_devs, nitems(bge_devs) - 1); DRIVER_MODULE(miibus, bge, miibus_driver, miibus_devclass, 0, 0); -static int bge_allow_asf = 1; +static int bge_allow_asf = 0; static SYSCTL_NODE(_hw, OID_AUTO, bge, CTLFLAG_RD, 0, "BGE driver parameters"); SYSCTL_INT(_hw_bge, OID_AUTO, allow_asf, CTLFLAG_RDTUN, &bge_allow_asf, 0, -- 2.32.0 From f538fa89af8dcda50c9f201aef3f1af37f4c6c31 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 7 Jan 2015 21:25:47 +0100 Subject: [PATCH 025/325] Assign random IP id values by default so users don't have to clown around with the sysctl themselves Obtained from: ElectroBSD --- sys/netinet/ip_id.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/netinet/ip_id.c b/sys/netinet/ip_id.c index d124d541442a..752dc3f62327 100644 --- a/sys/netinet/ip_id.c +++ b/sys/netinet/ip_id.c @@ -138,8 +138,8 @@ static void ipid_sysuninit(void); SYSCTL_DECL(_net_inet_ip); SYSCTL_PROC(_net_inet_ip, OID_AUTO, random_id, CTLTYPE_INT | CTLFLAG_VNET | CTLFLAG_RW, - &VNET_NAME(ip_do_randomid), 0, sysctl_ip_randomid, "IU", - "Assign random ip_id values"); + &VNET_NAME(ip_do_randomid), 1, sysctl_ip_randomid, "IU", + "Assign random ip_id values. Important for Tor relays and a good idea in general."); SYSCTL_INT(_net_inet_ip, OID_AUTO, rfc6864, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_rfc6864), 0, "Use constant IP ID for atomic datagrams"); -- 2.32.0 From 5b246ae583d44de64783114d38974c5201a33aff Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 11 Feb 2015 12:19:24 +0100 Subject: [PATCH 026/325] Add vendor copyright ... after putting on my robe and "police educational technican" hat. This commit is optional. Feel free to import any other ElectroBSD commit without including this one. Obtained from: ElectroBSD --- sys/sys/copyright.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/sys/copyright.h b/sys/sys/copyright.h index b34d6d20de23..7aa81dc4bb2f 100644 --- a/sys/sys/copyright.h +++ b/sys/sys/copyright.h @@ -32,7 +32,7 @@ /* Add a FreeBSD vendor copyright here */ #define COPYRIGHT_Vendor \ - "" + "Copyright (c) 2010-2020 Fabian Keil - IT-Beratung und Polizei-Erziehung\n" /* FreeBSD */ #define COPYRIGHT_FreeBSD \ -- 2.32.0 From 844182b754512bf02ff0c8d72d1c3a318c4277d2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 Jan 2016 13:45:32 +0100 Subject: [PATCH 027/325] mdocml: Change OS name used in man page headers to ElectroBSD Obtained from: ElectroBSD --- contrib/mandoc/msec.in | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/contrib/mandoc/msec.in b/contrib/mandoc/msec.in index d5b319f16a9f..3a46f7d8ca9d 100644 --- a/contrib/mandoc/msec.in +++ b/contrib/mandoc/msec.in @@ -22,13 +22,13 @@ * Be sure to escape strings. */ -LINE("1", "FreeBSD General Commands Manual") -LINE("2", "FreeBSD System Calls Manual") -LINE("3", "FreeBSD Library Functions Manual") +LINE("1", "ElectroBSD General Commands Manual") +LINE("2", "ElectroBSD System Calls Manual") +LINE("3", "ElectroBSD Library Functions Manual") LINE("3p", "Perl Library Functions Manual") -LINE("4", "FreeBSD Kernel Interfaces Manual") -LINE("5", "FreeBSD File Formats Manual") -LINE("6", "FreeBSD Games Manual") -LINE("7", "FreeBSD Miscellaneous Information Manual") -LINE("8", "FreeBSD System Manager\'s Manual") -LINE("9", "FreeBSD Kernel Developer\'s Manual") +LINE("4", "ElectroBSD Kernel Interfaces Manual") +LINE("5", "ElectroBSD File Formats Manual") +LINE("6", "ElectroBSD Games Manual") +LINE("7", "ElectroBSD Miscellaneous Information Manual") +LINE("8", "ElectroBSD System Manager\'s Manual") +LINE("9", "ElectroBSD Kernel Developer\'s Manual") -- 2.32.0 From 98349b6d9a14c7a9c1d31d413a71de4eb8299520 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 24 Jan 2016 20:24:00 +0100 Subject: [PATCH 028/325] clang: Set CLANG_VENDOR to ElectroBSD Obtained from: ElectroBSD --- lib/clang/include/clang/Basic/Version.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/clang/include/clang/Basic/Version.inc b/lib/clang/include/clang/Basic/Version.inc index 7114bbf5fd6e..7eb8dd064a4e 100644 --- a/lib/clang/include/clang/Basic/Version.inc +++ b/lib/clang/include/clang/Basic/Version.inc @@ -6,4 +6,4 @@ #define CLANG_VERSION_MINOR 0 #define CLANG_VERSION_PATCHLEVEL 1 -#define CLANG_VENDOR "FreeBSD " +#define CLANG_VENDOR "ElectroBSD " -- 2.32.0 From 42c8505a252e667286773a7da1c223f3c6c80e06 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 24 Feb 2015 19:35:03 +0100 Subject: [PATCH 029/325] Let rc.d/motd work with unames other than FreeBSD Obtained from: ElectroBSD --- libexec/rc/rc.d/motd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libexec/rc/rc.d/motd b/libexec/rc/rc.d/motd index acb376723e80..11d2257aa9c0 100755 --- a/libexec/rc/rc.d/motd +++ b/libexec/rc/rc.d/motd @@ -35,7 +35,7 @@ motd_start() T=`mktemp -t motd` uname -v | sed -e 's,^\([^#]*\) #\(.* [1-2][0-9][0-9][0-9]\).*/\([^\]*\) $,\1 (\3) #\2,' > ${T} - awk '{if (NR == 1) {if ($1 == "FreeBSD") {next} else {print "\n"$0}} else {print}}' < /etc/motd >> ${T} + awk '{if (NR == 1) {if ($1 == "'"$(uname)"'") {next} else {print "\n"$0}} else {print}}' < /etc/motd >> ${T} cmp -s $T /etc/motd || { cp $T /etc/motd -- 2.32.0 From 03e8c9eb74c805e6e063831d7eed94479e3a69bf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 4 May 2015 18:48:20 +0200 Subject: [PATCH 030/325] g_multipath: Add sysctl to disable tasting Obtained from: ElectroBSD --- sys/geom/multipath/g_multipath.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sys/geom/multipath/g_multipath.c b/sys/geom/multipath/g_multipath.c index 7a8c88ab61fc..66f20f80d238 100644 --- a/sys/geom/multipath/g_multipath.c +++ b/sys/geom/multipath/g_multipath.c @@ -61,6 +61,9 @@ SYSCTL_UINT(_kern_geom_multipath, OID_AUTO, debug, CTLFLAG_RW, static u_int g_multipath_exclusive = 1; SYSCTL_UINT(_kern_geom_multipath, OID_AUTO, exclusive, CTLFLAG_RW, &g_multipath_exclusive, 0, "Exclusively open providers"); +static u_int g_multipath_enable_tasting = 1; +SYSCTL_UINT(_kern_geom_multipath, OID_AUTO, taste, CTLFLAG_RW, + &g_multipath_enable_tasting, 0, "Enable multipath tasting. May cause conflicts."); SDT_PROVIDER_DECLARE(geom); SDT_PROBE_DEFINE2(geom, multipath, config, restore, "char*", "char*"); @@ -824,6 +827,9 @@ g_multipath_taste(struct g_class *mp, struct g_provider *pp, int flags __unused) g_topology_assert(); + if (g_multipath_enable_tasting == 0) + return (NULL); + gp = g_new_geomf(mp, "multipath:taste"); gp->start = g_multipath_start; gp->access = g_multipath_access; -- 2.32.0 From ed4aab58a004e1f707b84163bc096f10af44e6ed Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 9 May 2015 14:32:31 +0200 Subject: [PATCH 031/325] motd: Customize for ElectroBSD Obtained from: ElectroBSD --- usr.bin/login/motd | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/usr.bin/login/motd b/usr.bin/login/motd index 4dc41a744c31..3069d8ed4b20 100644 --- a/usr.bin/login/motd +++ b/usr.bin/login/motd @@ -1,21 +1,8 @@ -FreeBSD ?.?.? (UNKNOWN) +ElectroBSD ?.?.? (UNKNOWN) -Welcome to FreeBSD! +Welcome to ElectroBSD! -Release Notes, Errata: https://www.FreeBSD.org/releases/ -Security Advisories: https://www.FreeBSD.org/security/ -FreeBSD Handbook: https://www.FreeBSD.org/handbook/ -FreeBSD FAQ: https://www.FreeBSD.org/faq/ -Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/ -FreeBSD Forums: https://forums.FreeBSD.org/ - -Documents installed with the system are in the /usr/local/share/doc/freebsd/ -directory, or can be installed later with: pkg install en-freebsd-doc -For other languages, replace "en" with a language code like de or fr. - -Show the version of FreeBSD installed: freebsd-version ; uname -a -Please include that output and any error messages when posting questions. -Introduction to manual pages: man man -FreeBSD directory layout: man hier +For details see: +https://www.ElectroBSD.org/ Edit /etc/motd to change this login announcement. -- 2.32.0 From 29e1574a0c07d897895ffb392d94b40e2e9a0290 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 May 2015 14:18:43 +0200 Subject: [PATCH 032/325] release: Load usb modules through loader.conf so 'memstick' works with ELECTRO_BEER Obtained from: ElectroBSD --- release/Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/release/Makefile b/release/Makefile index 09b1b6a2c855..eec4279260f7 100644 --- a/release/Makefile +++ b/release/Makefile @@ -213,6 +213,9 @@ dvd: packagesystem echo hostid_enable=\"NO\" >> ${.TARGET}/etc/rc.conf echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf + for module in usb usb_quirk ehci umass; do \ + echo $${module}_load=\"YES\" >> ${.TARGET}/boot/loader.conf + done cp ${.CURDIR}/rc.local ${.TARGET}/etc touch ${.TARGET} -- 2.32.0 From d21262f2bbf820603f99adc41864346a6bd6520b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 May 2015 19:13:51 +0200 Subject: [PATCH 033/325] brand-fbsd.4th: Change OS name in banner to ElectroBSD While at it, suggest to resist unlawful police activities (German). Obtained from: ElectroBSD --- stand/forth/brand-fbsd.4th | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/stand/forth/brand-fbsd.4th b/stand/forth/brand-fbsd.4th index 9cd017f84a3f..0a62e98dc4d0 100644 --- a/stand/forth/brand-fbsd.4th +++ b/stand/forth/brand-fbsd.4th @@ -32,15 +32,17 @@ 1+ \ increase y for next time we're called ; -: brand ( x y -- ) \ "FreeBSD" [wide] logo in B/W (7 rows x 42 columns) +: brand ( x y -- ) \ "ElectroBSD" [wide] logo in B/W - s" ______ ____ _____ _____ " brand+ - s" | ____| | _ \ / ____| __ \ " brand+ - s" | |___ _ __ ___ ___ | |_) | (___ | | | |" brand+ - s" | ___| '__/ _ \/ _ \| _ < \___ \| | | |" brand+ - s" | | | | | __/ __/| |_) |____) | |__| |" brand+ - s" | | | | | | || | | |" brand+ - s" |_| |_| \___|\___||____/|_____/|_____/ " brand+ + s" ______ _ _ ____ _____ _____" brand+ + s" | ____| | | | | _ \ / ____| __ \" brand+ + s" | |__ | | ___ ___| |_ _ __ ___ | |_) | (___ | | | |" brand+ + s" | __| | |/ _ \/ __| __| '__/ _ \| _ < \___ \| | | |" brand+ + s" | |____| | __/ (__| |_| | | (_) | |_) |____) | |__| |" brand+ + s" |______|_|\___|\___|\__|_| \___/|____/|_____/|_____/" brand+ + s" Polizei-Willkuer in Deinem Land? Das erfordert Widerstand!" brand+ + s" Zu Risiken und Nebenwirkungen fragen Sie Ihren Anwalt oder" brand+ + s" die Rote Hilfe." brand+ 2drop ; -- 2.32.0 From 4eb1be03a01230999643bea1f9855ac859f760ae Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 May 2015 19:25:56 +0200 Subject: [PATCH 034/325] beastie.4th: Think of the children and default to showing beastie instead of the sex toy Obtained from: ElectroBSD --- stand/forth/beastie.4th | 4 ++-- stand/forth/beastie.4th.8 | 5 ++--- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/stand/forth/beastie.4th b/stand/forth/beastie.4th index 752cce22a4ff..6a28605bcefd 100644 --- a/stand/forth/beastie.4th +++ b/stand/forth/beastie.4th @@ -64,9 +64,9 @@ variable logoY s" loader_logo" getenv dup -1 = over 0= or if dup 0= if 2drop else drop then \ getenv result unused loader_color? if - s" try-include /boot/logo-orb.4th" + s" try-include /boot/logo-beastie.4th" else - s" try-include /boot/logo-orbbw.4th" + s" try-include /boot/logo-beastiebw.4th" then else 2drop ( c-addr/u -- ) \ getenv result unused diff --git a/stand/forth/beastie.4th.8 b/stand/forth/beastie.4th.8 index e99f65474b43..2cae805d95bd 100644 --- a/stand/forth/beastie.4th.8 +++ b/stand/forth/beastie.4th.8 @@ -106,10 +106,9 @@ The environment variables that effect its behavior are: Selects the desired logo in the beastie boot menu. Possible values are: .Dq Li fbsdbw , .Dq Li beastie , -.Dq Li beastiebw , +.Dq Li beastiebw (default) , .Dq Li orb , -.Dq Li orbbw -(default), and +.Dq Li orbbw , and .Dq Li none . .It Va loader_logo_x Sets the desired column position of the logo. Default is 46. -- 2.32.0 From 0d9b2968d679fbb4100c4305738f03109cd63fe1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 May 2015 19:39:56 +0200 Subject: [PATCH 035/325] menu.4th: Reduce visual noise by ditching the welcome text Obtained from: ElectroBSD --- stand/forth/menu.4th | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/forth/menu.4th b/stand/forth/menu.4th index 73eb88c1ef0c..73d25e1f3f0e 100644 --- a/stand/forth/menu.4th +++ b/stand/forth/menu.4th @@ -470,7 +470,7 @@ also menu-infrastructure definitions \ Print the frame caption at (x,y) s" loader_menu_title" getenv dup -1 = if - drop s" Welcome to FreeBSD" + drop s" " then TRUE ( use default alignment ) s" loader_menu_title_align" getenv dup -1 <> if -- 2.32.0 From 642c86a854555a3c90170f946974f33976a66465 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 May 2015 17:40:22 +0200 Subject: [PATCH 036/325] logo-beastiebw: Change the fork to a toilet brush Obtained from: ElectroBSD --- stand/forth/logo-beastiebw.4th | 40 +++++++++++++++++----------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/stand/forth/logo-beastiebw.4th b/stand/forth/logo-beastiebw.4th index 197099cda0bc..bce43f5b9d52 100644 --- a/stand/forth/logo-beastiebw.4th +++ b/stand/forth/logo-beastiebw.4th @@ -33,27 +33,27 @@ 1+ \ increase y for next time we're called ; -: logo ( x y -- ) \ B/W BSD mascot (19 rows x 34 columns) +: logo ( x y -- ) \ B/W BSD mascot with toilet brush - s" , ," logo+ - s" /( )`" logo+ - s" \ \___ / |" logo+ - s" /- _ `-/ '" logo+ - s" (/\/ \ \ /\" logo+ - s" / / | ` \" logo+ - s" O O ) / |" logo+ - s" `-^--'`< '" logo+ - s" (_.) _ ) /" logo+ - s" `.___/` /" logo+ - s" `-----' /" logo+ - s" <----. __ / __ \" logo+ - s" <----|====O)))==) \) /====|" logo+ - s" <----' `--' `.__,' \" logo+ - s" | |" logo+ - s" \ / /\" logo+ - s" ______( (_ / \______/" logo+ - s" ,' ,-----' |" logo+ - s" `--{__________)" logo+ + s" , ," logo+ + s" /( )`" logo+ + s" \ \___ / |" logo+ + s" /- _ `-/ '" logo+ + s" (/\/ \ \ /\" logo+ + s" / / | ` \" logo+ + s" O O ) / |" logo+ + s" `-^--'`< '" logo+ + s" (_.) _ ) /" logo+ + s" `.___/` /" logo+ + s" `-----' /" logo+ + s" ###### __ / __ \" logo+ + s" ######===O)))==) \) /====|" logo+ + s" ###### `--' `.__,' \" logo+ + s" | |" logo+ + s" \ / /\" logo+ + s" ______( (_ / \___/" logo+ + s" ,' ,-----' |" logo+ + s" `--{__________)" logo+ 2drop ; -- 2.32.0 From feda2b349090d8eed405ef9219f076c202508380 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 May 2015 18:19:19 +0200 Subject: [PATCH 037/325] logo-beastie.4th: Replace fork with toilet brush Obtained from: ElectroBSD --- stand/forth/logo-beastie.4th | 40 ++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/stand/forth/logo-beastie.4th b/stand/forth/logo-beastie.4th index 671eb5e496b2..8441c9ddafbd 100644 --- a/stand/forth/logo-beastie.4th +++ b/stand/forth/logo-beastie.4th @@ -35,27 +35,27 @@ 1+ \ increase y for next time we're called ; -: logo ( x y -- ) \ color BSD mascot (19 rows x 34 columns) +: logo ( x y -- ) \ color BSD mascot with toilet brush - s" @[31m, ," logo+ - s" /( )`" logo+ - s" \ \___ / |" logo+ - s" /- @[m_@[31m `-/ '" logo+ - s" (@[m/\/ \@[31m \ /\" logo+ - s" @[m/ / |@[31m ` \" logo+ - s" @[34mO O @[m) @[31m/ |" logo+ - s" @[m`-^--'@[31m`< '" logo+ - s" (_.) _ ) /" logo+ - s" `.___/` /" logo+ - s" `-----' /" logo+ - s" @[33m<----.@[31m __ / __ \" logo+ - s" @[33m<----|====@[31mO)))@[33m==@[31m) \) /@[33m====|" logo+ - s" @[33m<----'@[31m `--' `.__,' \" logo+ - s" | |" logo+ - s" \ / /\" logo+ - s" @[36m______@[31m( (_ / \______/" logo+ - s" @[36m,' ,-----' |" logo+ - s" `--{__________)@[m" logo+ + s" @[31m, ," logo+ + s" /( )`" logo+ + s" \ \___ / |" logo+ + s" /- @[m_@[31m `-/ '" logo+ + s" (@[m/\/ \@[31m \ /\" logo+ + s" @[m/ / |@[31m ` \" logo+ + s" @[34mO O @[m) @[31m/ |" logo+ + s" @[m`-^--'@[31m`< '" logo+ + s" (_.) _ ) /" logo+ + s" `.___/` /" logo+ + s" `-----' /" logo+ + s" @[37m######@[31m __ / __ \" logo+ + s" @[37m######====@[31mO)))@[37m==@[31m) \) /@[37m====|" logo+ + s" @[37m######@[31m `--' `.__,' \" logo+ + s" | |" logo+ + s" \ / /\" logo+ + s" @[36m______@[31m( (_ / \_____/" logo+ + s" @[36m,' ,-----' |" logo+ + s" `--{__________)@[m" logo+ 2drop ; -- 2.32.0 From d9515b39e928ba6c0879e64d6d204977561aa451 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 14 May 2015 10:57:01 +0200 Subject: [PATCH 038/325] rc.d/jail: Remove obnoxious warning about 'obsolete' jail_* variables They will not be removed from ElectroBSD until a replacement exists that is usable with shell scripts like ezjail without jumping through lots of hoops. Obtained from: ElectroBSD --- libexec/rc/rc.d/jail | 1 - 1 file changed, 1 deletion(-) diff --git a/libexec/rc/rc.d/jail b/libexec/rc/rc.d/jail index ff4336131bc9..3b24af3a818f 100755 --- a/libexec/rc/rc.d/jail +++ b/libexec/rc/rc.d/jail @@ -15,7 +15,6 @@ desc="Manage system jails" rcvar="jail_enable" start_cmd="jail_start" -start_postcmd="jail_warn" stop_cmd="jail_stop" config_cmd="jail_config" console_cmd="jail_console" -- 2.32.0 From edb1aeb5f7581b582bdfcf34072905a7942a7a96 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 14 May 2015 12:34:11 +0200 Subject: [PATCH 039/325] kern.opts.mk: Disable IPFILTER, IPSEC_SUPPORT, OFED, SOURCELESS_HOST and SOURCELESS_UCODE (XXX) This commit should probably be split into three. src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- sys/conf/kern.opts.mk | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/sys/conf/kern.opts.mk b/sys/conf/kern.opts.mk index e4334ce37237..8abf76de7068 100644 --- a/sys/conf/kern.opts.mk +++ b/sys/conf/kern.opts.mk @@ -34,19 +34,13 @@ __DEFAULT_YES_OPTIONS = \ FORMAT_EXTENSIONS \ INET \ INET6 \ - IPFILTER \ - IPSEC_SUPPORT \ ISCSI \ KERNEL_SYMBOLS \ MODULE_DRM \ MODULE_DRM2 \ NETGRAPH \ - OFED \ PF \ REPRODUCIBLE_BUILD \ - SCTP_SUPPORT \ - SOURCELESS_HOST \ - SOURCELESS_UCODE \ TESTS \ USB_GADGET_EXAMPLES \ ZFS @@ -54,8 +48,14 @@ __DEFAULT_YES_OPTIONS = \ __DEFAULT_NO_OPTIONS = \ EXTRA_TCP_STACKS \ KERNEL_RETPOLINE \ + IPFILTER \ + IPSEC_SUPPORT \ NAND \ - RATELIMIT + OFED \ + RATELIMIT \ + REPRODUCIBLE_BUILD \ + SOURCELESS_HOST \ + SOURCELESS_UCODE # Some options are totally broken on some architectures. We disable # them. If you need to enable them on an experimental basis, you -- 2.32.0 From 84506a5e297fd892d060a65367eee059360d5bbb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 15 May 2015 13:11:57 +0200 Subject: [PATCH 040/325] boot/newvers.sh branding: Change bootprog_name[] to ElectroBSD Obtained from: ElectroBSD --- stand/common/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/common/newvers.sh b/stand/common/newvers.sh index 714adba6c9cb..f5e7a04c40cf 100755 --- a/stand/common/newvers.sh +++ b/stand/common/newvers.sh @@ -50,7 +50,7 @@ u=${USER-root} h=${HOSTNAME-`hostname`} t=`date` #r=`head -n 6 $1 | tail -n 1 | awk -F: ' { print $1 } '` r=`awk -F: ' /^[0-9]\.[0-9]+:/ { print $1; exit }' $1` -bootprog_info="FreeBSD/${3} ${2}, Revision ${r}\\n" +bootprog_info="ElectroBSD/${3} ${2}, Revision ${r}\\n" if [ -n "${include_metadata}" ]; then bootprog_info="$bootprog_info(${t} ${u}@${h})\\n" fi -- 2.32.0 From 7a73950e29e435ae6894bc29af852ea7291bc2ab Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 18 May 2015 15:59:58 +0200 Subject: [PATCH 041/325] release: Default to not distributing the ports tree ... as there are too many license problems. Allow the user to set WITH_PORTS to distribute the ports tree anyway. Obtained from: ElectroBSD --- release/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/release/Makefile b/release/Makefile index eec4279260f7..06f984086dcc 100644 --- a/release/Makefile +++ b/release/Makefile @@ -22,8 +22,8 @@ # PORTSDIR: location of ports tree to distribute (default: /usr/ports) # XTRADIR: xtra-bits-dir argument for /mkisoimages.sh # NOPKG: if set, do not distribute third-party packages -# NOPORTS: if set, do not distribute ports tree # NOSRC: if set, do not distribute source tree +# WITH_PORTS: if set, distribute ports tree provided it exists # WITH_DVD: if set, generate dvd1.iso # WITH_COMPRESSED_IMAGES: if set, compress installation images with xz(1) # (uncompressed images are not removed) @@ -71,7 +71,7 @@ VOLUME_LABEL= ${REVISION:C/[.-]/_/g}_${BRANCH:C/[.-]/_/g}_${TARGET_ARCH} VOLUME_LABEL= FreeBSD_Install .endif -.if !exists(${PORTSDIR}) +.if !exists(${PORTSDIR}) || !defined(WITH_PORTS) NOPORTS= true .endif -- 2.32.0 From 3b72c2616bc58e6a1dd702af5600a714efe20496 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 May 2015 15:26:44 +0200 Subject: [PATCH 042/325] sys/boot/common/newvers.sh: Allow to overwrite the date to make boot loader binaries reproducible Obtained from: ElectroBSD --- stand/common/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/common/newvers.sh b/stand/common/newvers.sh index f5e7a04c40cf..e1d17e57d23c 100755 --- a/stand/common/newvers.sh +++ b/stand/common/newvers.sh @@ -46,7 +46,7 @@ done shift $((OPTIND - 1)) LC_ALL=C; export LC_ALL -u=${USER-root} h=${HOSTNAME-`hostname`} t=`date` +u=${USER-root} h=${HOSTNAME-`hostname`} t=${DATE-`date`} #r=`head -n 6 $1 | tail -n 1 | awk -F: ' { print $1 } '` r=`awk -F: ' /^[0-9]\.[0-9]+:/ { print $1; exit }' $1` -- 2.32.0 From 93e1f3576aaa4105acfa3bb377fd38a0ffecf8c9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 May 2015 15:36:48 +0200 Subject: [PATCH 043/325] sys/conf/newvers.sh: Allow to overwrite the build date embedded into the kernel This is a required step to get reproducible builds. Obtained from: ElectroBSD --- sys/conf/newvers.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index bc3a496106ec..11025ba4cde2 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -133,7 +133,12 @@ v=${KERNEL_VERSION_NUMBER:-`cat version`} u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} -if [ -n "$SOURCE_DATE_EPOCH" ]; then +if [ -n "${DATE}" ]; then + # SOURCE_DATE_EPOCH was added upstream in r291691 + # but the ElectroBSD build goo is still setting DATE + # which has a different format. + t=${DATE} +elif [ -n "$SOURCE_DATE_EPOCH" ]; then if ! t=`date -r $SOURCE_DATE_EPOCH 2>/dev/null`; then echo "Invalid SOURCE_DATE_EPOCH" >&2 exit 1 -- 2.32.0 From 62c43b79b2b32e6bad4efaff7d26e6f4250a5ab7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 19 May 2015 16:06:01 +0200 Subject: [PATCH 044/325] release/amd64/make-memstick.sh: Use reproducible timestamps for the makefs image Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 3de8cb2a0b83..3dd87d12d0c2 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -32,9 +32,11 @@ fi echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local -makefs -B little -o label=FreeBSD_Install -o version=2 ${2}.part ${1} +mtree -c -k time -p "${1}" | sed "s@time=.*@time=${EPOCH_DATE-0}.000000000@" > "${2}.mtree" +makefs -B little -o label=FreeBSD_Install -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local +rm "${2}.mtree" mkimg -s mbr \ -b ${1}/boot/mbr \ -- 2.32.0 From 465499155fe5a40db251f0053bb4d022294b18b8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 17 Sep 2015 11:53:45 +0200 Subject: [PATCH 045/325] make-memstick.sh: Additionally fake uid and gid on the created fs Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 3dd87d12d0c2..c4f4619f4408 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -32,7 +32,12 @@ fi echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local -mtree -c -k time -p "${1}" | sed "s@time=.*@time=${EPOCH_DATE-0}.000000000@" > "${2}.mtree" +# Prepare mtree spec to fake timestamp, owner and group. +# As a result, man pages will be owned by root instead of man. +# Unfortunately we can't simply reuse ${1}/METALOG as it is incomplete. +mtree -c -k time -p "${1}" | sed \ + -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ + > "${2}.mtree" || return 1 makefs -B little -o label=FreeBSD_Install -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local -- 2.32.0 From 3a84c7c71aea149e969e253c6f99fd23a6246ff2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 30 Sep 2015 13:15:44 +0200 Subject: [PATCH 046/325] release/amd64/make-memstick.sh: Error out if mkimg fails instead of cleaning up Makes debugging more convenient. Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index c4f4619f4408..338a8ad18b62 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -49,5 +49,9 @@ mkimg -s mbr \ -p freebsd:-"mkimg -s bsd -b ${1}/boot/boot -p freebsd-ufs:=${2}.part" \ -a 2 \ -o ${2} +if [ $? -ne 0 ]; then + echo "mkimg failed" + exit 1 +fi rm ${2}.part -- 2.32.0 From 6807feb7d2974c67141c5f0b07ccf209aff88cb9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 21 May 2015 18:40:56 +0200 Subject: [PATCH 047/325] release/amd64/make-memstick.sh: Allow to overwrite VOLUME_LABEL through the environment Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 338a8ad18b62..2db5401d89ec 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -14,6 +14,7 @@ set -e PATH=/bin:/usr/bin:/sbin:/usr/sbin export PATH +VOLUME_LABEL=${VOLUME_LABEL-"FreeBSD_Install"} if [ $# -ne 2 ]; then echo "make-memstick.sh /path/to/directory /path/to/image/file" @@ -30,7 +31,7 @@ if [ -e ${2} ]; then exit 1 fi -echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab +echo "/dev/ufs/${VOLUME_LABEL} / ufs ro,noatime 1 1" > ${1}/etc/fstab echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local # Prepare mtree spec to fake timestamp, owner and group. # As a result, man pages will be owned by root instead of man. @@ -38,7 +39,7 @@ echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local mtree -c -k time -p "${1}" | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 -makefs -B little -o label=FreeBSD_Install -o version=2 -F "${2}.mtree" ${2}.part ${1} +makefs -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local rm "${2}.mtree" -- 2.32.0 From 204fd516bb388690a9b73ac38f5552028f94875a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 28 Jun 2015 13:45:09 +0200 Subject: [PATCH 048/325] Use make-memstick.sh's amd64 version on i386 as well Obtained from: ElectroBSD --- release/i386/make-memstick.sh | 45 +---------------------------------- 1 file changed, 1 insertion(+), 44 deletions(-) mode change 100755 => 120000 release/i386/make-memstick.sh diff --git a/release/i386/make-memstick.sh b/release/i386/make-memstick.sh deleted file mode 100755 index 6774e86ae550..000000000000 --- a/release/i386/make-memstick.sh +++ /dev/null @@ -1,44 +0,0 @@ -#!/bin/sh -# -# This script generates a "memstick image" (image that can be copied to a -# USB memory stick) from a directory tree. Note that the script does not -# clean up after itself very well for error conditions on purpose so the -# problem can be diagnosed (full filesystem most likely but ...). -# -# Usage: make-memstick.sh -# -# $FreeBSD$ -# - -set -e - -PATH=/bin:/usr/bin:/sbin:/usr/sbin -export PATH - -if [ $# -ne 2 ]; then - echo "make-memstick.sh /path/to/directory /path/to/image/file" - exit 1 -fi - -if [ ! -d ${1} ]; then - echo "${1} must be a directory" - exit 1 -fi - -if [ -e ${2} ]; then - echo "won't overwrite ${2}" - exit 1 -fi - -echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab -echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local -makefs -B little -o label=FreeBSD_Install -o version=2 ${2}.part ${1} -rm ${1}/etc/fstab -rm ${1}/etc/rc.conf.local - -mkimg -s mbr \ - -b ${1}/boot/mbr \ - -p freebsd:-"mkimg -s bsd -b ${1}/boot/boot -p freebsd-ufs:=${2}.part" \ - -o ${2} -rm ${2}.part - diff --git a/release/i386/make-memstick.sh b/release/i386/make-memstick.sh new file mode 120000 index 000000000000..978e04a02184 --- /dev/null +++ b/release/i386/make-memstick.sh @@ -0,0 +1 @@ +../amd64/make-memstick.sh \ No newline at end of file -- 2.32.0 From 7582357af8c11268ed2873f02d51f325cd1e97fb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 2 Feb 2016 20:28:35 +0100 Subject: [PATCH 049/325] release/amd64/make-memstick.sh: Use the shiny new -T switch to clamp the remaining timestamps Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 2db5401d89ec..5025e12fe037 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -39,7 +39,10 @@ echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local mtree -c -k time -p "${1}" | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 -makefs -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} +# The base makefs may not have -T support yet, +# thus we use the fresh one that goes into the image. +dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ + -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local rm "${2}.mtree" -- 2.32.0 From 182baed266c2f06b41455d867c4c9b30bcc93d27 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 21 May 2017 18:10:15 +0200 Subject: [PATCH 050/325] release/amd64/make-memstick.sh: Attempt to build the mtree spec reproducible Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 5025e12fe037..6476cf812a29 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -36,7 +36,7 @@ echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local # Prepare mtree spec to fake timestamp, owner and group. # As a result, man pages will be owned by root instead of man. # Unfortunately we can't simply reuse ${1}/METALOG as it is incomplete. -mtree -c -k time -p "${1}" | sed \ +mtree -c -k time -p "${1}" | mtree -C -k all | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 # The base makefs may not have -T support yet, -- 2.32.0 From 788c4e8c780cf9454800ff93c875d4994fbc3fc3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 21 May 2017 17:58:45 +0200 Subject: [PATCH 051/325] release/amd64/make-memstick.sh: Limit makefs to a single cpu ... to see if it's sufficient to get reproducible images without having to limit the number of cores for everything. Spoiler alert: Nope. Currently the image checksum is the same when using 1, 2 or 3 cores but changes when using 4: fk@t520 ~ $diffoscope-3.5 /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img --- /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img +++ /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img @@ -7641,16 +7641,16 @@ 000428b0 53 44 2d 31 31 2e 31 2d 50 52 45 52 45 4c 45 41 |SD-11.1-PRERELEA| 000428b0 53 44 2d 31 31 2e 31 2d 50 52 45 52 45 4c 45 41 |SD-11.1-PRERELEA| 000428c0 53 45 2d 61 6d 64 36 00 00 00 00 00 00 00 00 00 |SE-amd6.........| 000428c0 53 45 2d 61 6d 64 36 00 00 00 00 00 00 00 00 00 |SE-amd6.........| 000428d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000428d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * -00042940 00 75 92 01 08 00 00 00 00 79 92 01 08 00 00 00 |.u.......y......| -00042940 00 75 92 01 08 00 00 00 00 79 92 01 08 00 00 00 |.u.......y......| +00042940 00 85 92 01 08 00 00 00 00 89 92 01 08 00 00 00 |................| +00042940 00 85 92 01 08 00 00 00 00 89 92 01 08 00 00 00 |................| 00042950 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..| 00042950 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..| 00042960 00 00 00 00 00 00 00 00 d8 ec 08 00 00 00 00 00 |................| 00042960 00 00 00 00 00 00 00 00 d8 ec 08 00 00 00 00 00 |................| 00042970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00042970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * @@ -7724,16 +7724,16 @@ 000448b0 53 44 2d 31 31 2e 31 2d 50 52 45 52 45 4c 45 41 |SD-11.1-PRERELEA| 000448b0 53 44 2d 31 31 2e 31 2d 50 52 45 52 45 4c 45 41 |SD-11.1-PRERELEA| 000448c0 53 45 2d 61 6d 64 36 00 00 00 00 00 00 00 00 00 |SE-amd6.........| 000448c0 53 45 2d 61 6d 64 36 00 00 00 00 00 00 00 00 00 |SE-amd6.........| 000448d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000448d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * -00044940 00 75 92 01 08 00 00 00 00 79 92 01 08 00 00 00 |.u.......y......| -00044940 00 75 92 01 08 00 00 00 00 79 92 01 08 00 00 00 |.u.......y......| +00044940 00 85 92 01 08 00 00 00 00 89 92 01 08 00 00 00 |................| +00044940 00 85 92 01 08 00 00 00 00 89 92 01 08 00 00 00 |................| 00044950 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..| 00044950 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..| 00044960 00 00 00 00 00 00 00 00 d8 ec 08 00 00 00 00 00 |................| 00044960 00 00 00 00 00 00 00 00 d8 ec 08 00 00 00 00 00 |................| 00044970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00044970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * @@ -1048569,8 +1048569,8 @@ 00871140 84 1e 81 ff 50 38 0f 31 89 04 25 2c 84 1e 81 66 |....P8.1..%,...f| 00871140 84 1e 81 ff 50 38 0f 31 89 04 25 2c 84 1e 81 66 |....P8.1..%,...f| 00871150 c7 04 25 38 84 1e 81 08 01 c6 04 25 3b 84 1e 81 |..%8.......%;...| 00871150 c7 04 25 38 84 1e 81 08 01 c6 04 25 3b 84 1e 81 |..%8.......%;...| 00871160 00 8b 04 25 28 84 1e 81 8d 48 01 89 0c 25 28 84 |...%(....H...%(.| 00871160 00 8b 04 25 28 84 1e 81 8d 48 01 89 0c 25 28 84 |...%(....H...%(.| 00871170 1e 81 88 04 25 3a 84 1e 81 48 c7 c7 30 84 1e 81 |....%:...H..0...| -[ Too much input for diff (SHA1: 6efa7af7add1fe5ba288c5ea10dd9e2328a4c697) ] +[ Too much input for diff (SHA1: 867b19a5d3489d643cdeb48f1950d1d5f6af4f93) ] The changing bytes are in the UFS partition but apparently invisible from above: fk@t520 ~ $sudo /usr/src/release/scripts/image-checksum.sh -v /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img Mounting /dev/md0p2 at /mnt Running mtree, saving spec in /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img.mtree Unmounting /mnt ... Mounting /dev/md0p2 at /mnt Running ls to get inodes, saving output in /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img.ls Unmounting /mnt ... gpart checksum: df93d1be80c1870b38e2e9888c32898f090c72aad8f72838f8ee9117f15a326e Boot code checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 mtree checksum: 758bff96cef51a0fbc8f0aa8386a7c00ef084740d7c185a6d9152a1a9f17e3a3 ls checksum: 2c2d99a9409f9eb1b0d32b0defa65f5454017e29e57f7295c02c495c121f0dd0 Partial image checksum for /images/ElectroBSD-r318528-2613fdfb3469-bhyve-2c/ElectroBSD-r318528-2613fdfb3469.img: dff88a8272549f8e01ad7cc325f5af284ba81f964bcc8001e373af287b5a4b05 Mounting /dev/md0p2 at /mnt Running mtree, saving spec in /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img.mtree Unmounting /mnt ... Mounting /dev/md0p2 at /mnt Running ls to get inodes, saving output in /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img.ls Unmounting /mnt ... gpart checksum: df93d1be80c1870b38e2e9888c32898f090c72aad8f72838f8ee9117f15a326e Boot code checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 mtree checksum: 758bff96cef51a0fbc8f0aa8386a7c00ef084740d7c185a6d9152a1a9f17e3a3 ls checksum: 2c2d99a9409f9eb1b0d32b0defa65f5454017e29e57f7295c02c495c121f0dd0 Partial image checksum for /images/ElectroBSD-r318528-2613fdfb3469-bhyve-4c/ElectroBSD-r318528-2613fdfb3469.img: dff88a8272549f8e01ad7cc325f5af284ba81f964bcc8001e373af287b5a4b05 While this commit doesn't actually solve anything, it's kept for now as it doesn't hurt and it's useful to keep an example difference around. Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 6476cf812a29..af3e5003bf76 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -41,7 +41,10 @@ mtree -c -k time -p "${1}" | mtree -C -k all | sed \ > "${2}.mtree" || return 1 # The base makefs may not have -T support yet, # thus we use the fresh one that goes into the image. -dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ +# +# Bind makefs to a single cpu as the output is core dependant +# (XXX: doesn't seem to have the intended effect) +cpuset -l 0 dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local -- 2.32.0 From 1b018ec55b2007297768141458d6c89e684ba535 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 May 2017 13:12:05 +0200 Subject: [PATCH 052/325] release/amd64/make-memstick.sh: Specify makefs debug flags which may or may not help Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index af3e5003bf76..c28521e9cbeb 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -39,12 +39,19 @@ echo 'root_rw_mount="NO"' > ${1}/etc/rc.conf.local mtree -c -k time -p "${1}" | mtree -C -k all | sed \ -e "s@time=.*@time=${EPOCH_DATE-0}.000000000 uname=root gname=wheel@" \ > "${2}.mtree" || return 1 + +#define DEBUG_FS_MAKEFS 0x00000400 +#define DEBUG_FS_CREATE_IMAGE 0x00001000 +#define DEBUG_FS_WRITE_FILE 0x00080000 +MAKEFS_DEBUG_FLAGS=0x81400 + # The base makefs may not have -T support yet, # thus we use the fresh one that goes into the image. # # Bind makefs to a single cpu as the output is core dependant # (XXX: doesn't seem to have the intended effect) cpuset -l 0 dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ + -d "${MAKEFS_DEBUG_FLAGS}" \ -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local -- 2.32.0 From 777a8e926dde0374d2afede0f3e119321d7920c9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 May 2015 18:45:24 +0200 Subject: [PATCH 053/325] release: Use a hack to recreate dist tarballs with reproducible timestamps (XXX: obsolete) Obtained from: ElectroBSD --- release/Makefile | 3 ++ release/scripts/tar-time-reset.sh | 64 +++++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+) create mode 100755 release/scripts/tar-time-reset.sh diff --git a/release/Makefile b/release/Makefile index 06f984086dcc..652a3f390556 100644 --- a/release/Makefile +++ b/release/Makefile @@ -238,6 +238,9 @@ mini-memstick.img: bootonly sh ${.CURDIR}/${TARGET}/make-memstick.sh bootonly ${.TARGET} packagesystem: base.txz kernel.txz ${EXTRA_PACKAGES} + for tarball in *.txz; do \ + sh ${.CURDIR}/scripts/tar-time-reset.sh $${tarball}; \ + done sh ${.CURDIR}/scripts/make-manifest.sh *.txz > MANIFEST touch ${.TARGET} diff --git a/release/scripts/tar-time-reset.sh b/release/scripts/tar-time-reset.sh new file mode 100755 index 000000000000..8186fd8d3e26 --- /dev/null +++ b/release/scripts/tar-time-reset.sh @@ -0,0 +1,64 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## +# +# This script resets the timestamps in a given tarfile to hopefully make +# it reproducible. This is a rather wasteful approach, but works for now. +# +# A better solution would be to patch bsdtar to optionally use a fixed +# time (without having to clown around with mtree specs). +# +########################################################################## + +main() { + local tarfile="${1}" \ + tempdir mtree_spec + + if [ $# -ne 1 ]; then + echo "$0 /path/to/tarfile" + exit 1 + fi + + if [ -z "${tarfile}" ]; then + echo "No tar file given" + return 1 + fi + tarfile="$(realpath "$tarfile")" + + tempdir=$(mktemp -d) || return 1 + mtree_spec=$(mktemp) || return 1 + + echo "Extracting tarfile ${tarfile}" + (cd "${tempdir}" && tar xvf "${tarfile}") || return 1 + + echo "Ditching original tarfile ${tarfile}" + rm "${tarfile}" + + echo "Creating mtree spec in ${mtree_spec}" + (cd "${tempdir}" && mtree -L -c -k time) | \ + sed "s@time=.*@time=${EPOCH_DATE-0}.000000000@" > "${mtree_spec}" + + echo "Creating tarfile ${tarfile}" + (cd "${tempdir}" && tar acLvf "${tarfile}" @"${mtree_spec}") || return 1 + + echo "Ditching ${tempdir}" + rm -r "${tempdir}" || return 1 + echo "Ditching ${mtree_spec}" + rm "${mtree_spec}" || return 1 +} + +main "${@}" -- 2.32.0 From 9ef12a34325eafe552b4c515dbe555fd5cf91944 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 22 May 2015 11:33:50 +0200 Subject: [PATCH 054/325] release/scripts/mm-mtree.sh: Allow to build the mtree spec for mergemaster reproducible This relies on NetBSD mtree which has been the default for a while now. Obtained from: ElectroBSD --- release/scripts/mm-mtree.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/release/scripts/mm-mtree.sh b/release/scripts/mm-mtree.sh index 790154595798..8c87bae03ab3 100755 --- a/release/scripts/mm-mtree.sh +++ b/release/scripts/mm-mtree.sh @@ -140,8 +140,12 @@ find ${TEMPROOT} -type f -size 0 -delete 2>/dev/null find -d ${TEMPROOT} -type d -empty -delete 2>/dev/null # Build the mtree database in a temporary location. +# The second mtree call is used to get a reproducible result +# without embedded hostname, user name and creation timestamp. +# XXX: Might be obsolete after r301584. MTREENEW=`mktemp -t mergemaster.mtree` -mtree -nci -p ${TEMPROOT} -k size,md5digest > ${MTREENEW} 2>/dev/null +mtree -mci -p ${TEMPROOT} -k size,md5digest 2>/dev/null | \ + mtree -C -k all > ${MTREENEW} if [ -s "${MTREENEW}" ]; then echo "*** Saving mtree database for future upgrades" -- 2.32.0 From ac77cdc72d9b3b12120f42ba2477ac6f1438b597 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 18 May 2015 19:17:14 +0200 Subject: [PATCH 055/325] release/Makefile: Don't create matroshka src tarballs that contain other tarballs ... if DESTDIR isn't set to a reasonable value. While at it, exclude *.orig files as well. Obtained from: ElectroBSD --- release/Makefile | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/release/Makefile b/release/Makefile index 652a3f390556..95a4c94cc04c 100644 --- a/release/Makefile +++ b/release/Makefile @@ -137,7 +137,9 @@ src.txz: mkdir -p ${DISTDIR}/usr ln -fs ${WORLDDIR} ${DISTDIR}/usr/src cd ${DISTDIR} && tar cLvf - --exclude .svn --exclude .zfs \ - --exclude .git --exclude @ --exclude usr/src/release/dist usr/src | \ + --exclude .git --exclude @ --exclude usr/src/release/dist \ + --exclude "usr/src/release/*.txz" \ + --exclude "usr/src/release/*.orig" usr/src | \ ${XZ_CMD} > ${.OBJDIR}/src.txz ports.txz: -- 2.32.0 From a7ae81ab1c41f29330bc934afe14c98c33cc663c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 14 May 2015 12:07:25 +0200 Subject: [PATCH 056/325] Make reproducing builds more convenient ... by setting the various variables based on the environment variable REPRO_SEED. Obtained from: ElectroBSD --- Makefile.inc1 | 14 ++++++-- release/Makefile | 2 ++ share/mk/src.reproducible-build.mk | 54 ++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+), 3 deletions(-) create mode 100644 share/mk/src.reproducible-build.mk diff --git a/Makefile.inc1 b/Makefile.inc1 index 1c9058342252..cbe03966dd4a 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -54,6 +54,8 @@ _MKSHOWCONFIG= t .endif +.include "share/mk/src.reproducible-build.mk" + SRCDIR?= ${.CURDIR} LOCALBASE?= /usr/local @@ -1183,7 +1185,7 @@ WMAKE_TGTS+= build${libcompat} buildworld: buildworld_prologue ${WMAKE_TGTS} buildworld_epilogue .PHONY .ORDER: buildworld_prologue ${WMAKE_TGTS} buildworld_epilogue -buildworld_prologue: .PHONY +buildworld_prologue: .PHONY reproducible_build_hint @echo "--------------------------------------------------------------" @echo ">>> World build started on `LC_ALL=C date`" .if ${TARGET:Mmips} @@ -1192,7 +1194,7 @@ buildworld_prologue: .PHONY .endif @echo "--------------------------------------------------------------" -buildworld_epilogue: .PHONY +buildworld_epilogue: .PHONY reproducible_build_hint @echo @echo "--------------------------------------------------------------" @echo ">>> World build completed on `LC_ALL=C date`" @@ -1202,6 +1204,12 @@ buildworld_epilogue: .PHONY .endif @echo "--------------------------------------------------------------" +reproducible_build_hint: + @echo "--------------------------------------------------------------" + @echo ">>> To reproduce this build:" + @echo ">>> export REPRO_SEED=$${REPRO_SEED}" + @echo "--------------------------------------------------------------" + # # We need to have this as a target because the indirection between Makefile # and Makefile.inc1 causes the correct PATH to be used, rather than a @@ -1658,7 +1666,7 @@ ${WMAKE_TGTS:N_worldtmp:Nbuild${libcompat}} ${.ALLTARGETS:M_*:N_worldtmp}: .MAKE # # Builds all kernels defined by BUILDKERNELS. # -buildkernel: .MAKE .PHONY +buildkernel: .MAKE .PHONY reproducible_build_hint .if empty(BUILDKERNELS:Ndummy) @echo "ERROR: Missing kernel configuration file(s) (${KERNCONF})."; \ false diff --git a/release/Makefile b/release/Makefile index 95a4c94cc04c..e029d100c125 100644 --- a/release/Makefile +++ b/release/Makefile @@ -34,6 +34,8 @@ # TARGET/TARGET_ARCH: architecture of built release # +.include "../share/mk/src.reproducible-build.mk" + WORLDDIR?= ${.CURDIR}/.. PORTSDIR?= /usr/ports RELNOTES_LANG?= en_US.ISO8859-1 diff --git a/share/mk/src.reproducible-build.mk b/share/mk/src.reproducible-build.mk new file mode 100644 index 000000000000..dc4f4df93507 --- /dev/null +++ b/share/mk/src.reproducible-build.mk @@ -0,0 +1,54 @@ +########################################################################## +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## +# +# Make the build reproducible by exporting a bunch of variables, +# potentionally using an already-set REPRO_SEED as input. +# +# The variable names are somewhat stupid, mostly because we +# are using existing ones. +# +########################################################################## + +TZ= "UTC" +.export TZ + +.if ! defined(REPRO_SEED) +# XXX: Currently we don't add the kernel version number to the repro +# seed because we expect a clean object tree in which case it +# will reproducible be 0. If the object tree of a the +# build-to-reproduce was actually unclean, KERNEL_VERSION_NUMBER +# has to be set to a matching value. +REPRO_SEED!= echo $$(id -un):$$(hostname):$$(date +%s) +.export REPRO_SEED +.else +.if ! defined(KERNEL_VERSION_NUMBER) +KERNEL_VERSION_NUMBER=0 +.export KERNEL_VERSION_NUMBER +.endif +.endif + +USER!= echo "${REPRO_SEED}" | /usr/bin/cut -d : -f 1 +HOSTNAME!= echo "${REPRO_SEED}" | /usr/bin/cut -d : -f 2 +EPOCH_DATE!= echo "${REPRO_SEED}" | /usr/bin/cut -d : -f 3 + +DATE!= date -r ${EPOCH_DATE} +# These two probably are no longer necessary after r285701 +MKREPRO_DATE!= date -r ${EPOCH_DATE} +"%b %d %Y" +MKREPRO_TIME!= date -r ${EPOCH_DATE} +%H:%M:%S + +.for v in REPRO_SEED USER HOSTNAME EPOCH_DATE DATE MKREPRO_DATE MKREPRO_TIME +.export $v +.endfor -- 2.32.0 From 9973bfbfc80c25de5a9a5d1b5f6b3137998cae80 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 24 May 2015 18:01:31 +0200 Subject: [PATCH 057/325] Fake modification time smarter for everything but EXTRA_PACKAGES (XXX) While at it, remove duplicated slashes in the METALOG as they result in missing files. XXX 2017-01: It's not clear if this is still the case, but embedding duplicated slashes in the dist files is also aesthetically unpleasing. Mark two suspicious mtree spec modifications as such. XXX: investigate. Obtained from: ElectroBSD --- Makefile.inc1 | 20 ++++++++++++++++++++ release/Makefile | 8 ++++++-- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/Makefile.inc1 b/Makefile.inc1 index cbe03966dd4a..74426f0984dd 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1453,6 +1453,14 @@ distributeworld installworld stageworld: _installcheck_world .PHONY find ${DESTDIR}/${DISTDIR}/${dist} -mindepth 1 -type d -empty -delete .endfor .if defined(NO_ROOT) + @# Post process METALOG: add fake timestamps and, if necessary, + @# remove duplicated slashes which can occur if DISTDIR is undefined. + @# Keeping them results in missing files in the distribution tarballs. +.if defined(EPOCH_DATE) + sed -E -e 's@time=[0-9\.]+@@' \ + -e 's@(type=)@uid=0 gid=0 time=${EPOCH_DATE}.0 \1@' \ + -e 's@//@/@g' -i '.bak' ${METALOG} +.endif .for dist in base ${EXTRA_DISTRIBUTIONS} @# For each file that exists in this dist, print the corresponding @# line from the METALOG. This relies on the fact that @@ -1800,6 +1808,12 @@ distributekernel distributekernel.debug: .PHONY packagekernel: .PHONY .if defined(NO_ROOT) .if !defined(NO_INSTALLKERNEL) +.if defined(EPOCH_DATE) +# XXX: Is this really necessary given that we already modify the METALOG itself? + sed -E -e 's@time=[0-9\.]+@@' -e 's@(type=)@time=${EPOCH_DATE}.0 \1@' \ + -e 's@//@/@g' \ + -i '.bak' ${DESTDIR}/${DISTDIR}/kernel.meta +.endif cd ${DESTDIR}/${DISTDIR}/kernel; \ tar cvf - --exclude '*.debug' \ @${DESTDIR}/${DISTDIR}/kernel.meta | \ @@ -1813,6 +1827,12 @@ packagekernel: .PHONY .endif .if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes" .for _kernel in ${BUILDKERNELS:[2..-1]} +# XXX: See XXX above +.if defined(EPOCH_DATE) + sed -E -e 's@time=[0-9\.]+@@' -e 's@(type=)@time=${EPOCH_DATE}.0 \1@' \ + -e 's@//@/@g' \ + -i '.bak' ${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta +.endif cd ${DESTDIR}/${DISTDIR}/kernel.${_kernel}; \ tar cvf - --exclude '*.debug' \ @${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta | \ diff --git a/release/Makefile b/release/Makefile index e029d100c125..ae839b3aaf07 100644 --- a/release/Makefile +++ b/release/Makefile @@ -53,7 +53,11 @@ TARGET_ARCH= ${TARGET} IMAKE= ${MAKE} TARGET_ARCH=${TARGET_ARCH} TARGET=${TARGET} DISTDIR= dist -# Define OSRELEASE by using newvers.sh +# Enable mtree spec usage so we can fake the modification time. +NO_ROOT=1 +.export NO_ROOT + +# Define OSRELEASE by using newvars.sh .if !defined(OSRELEASE) || empty(OSRELEASE) .for _V in TYPE BRANCH REVISION ${_V}!= eval $$(awk '/^${_V}=/{print}' ${.CURDIR}/../sys/conf/newvers.sh); echo $$${_V} @@ -242,7 +246,7 @@ mini-memstick.img: bootonly sh ${.CURDIR}/${TARGET}/make-memstick.sh bootonly ${.TARGET} packagesystem: base.txz kernel.txz ${EXTRA_PACKAGES} - for tarball in *.txz; do \ + for tarball in ${EXTRA_PACKAGES}; do \ sh ${.CURDIR}/scripts/tar-time-reset.sh $${tarball}; \ done sh ${.CURDIR}/scripts/make-manifest.sh *.txz > MANIFEST -- 2.32.0 From 03e596d43d15c177c3c72c388db7a7258b326949 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Jun 2015 13:56:05 +0200 Subject: [PATCH 058/325] release/Makefile: Build the src.txz only once ... and also enforce reproducible source ownership and permissions. If taring the sources fails, keep the mtree spec. Keep release/scripts/tar-time-reset.sh for now but update a comment to make it obvious that the script isn't used anymore. NB: A previos version of this commit gave directories in the source tarball 550 permissions. This seems reasonable at first glance but breaks poudriere builds with unprivileges users. Poudriere applies the permissions from the source tarball to /usr, thus making it unaccesible for users other than root and members of wheel. While only giving /usr 555 permissions is sufficient to prevent the issue, treating the directory is more work and as the source is public anyway it didn't seem worth it. Obtained from: ElectroBSD --- release/Makefile | 31 +++++++++++++++++++++++-------- release/scripts/tar-time-reset.sh | 8 +++----- 2 files changed, 26 insertions(+), 13 deletions(-) diff --git a/release/Makefile b/release/Makefile index ae839b3aaf07..d90074ada1cc 100644 --- a/release/Makefile +++ b/release/Makefile @@ -142,11 +142,29 @@ kernel.txz: src.txz: mkdir -p ${DISTDIR}/usr ln -fs ${WORLDDIR} ${DISTDIR}/usr/src - cd ${DISTDIR} && tar cLvf - --exclude .svn --exclude .zfs \ - --exclude .git --exclude @ --exclude usr/src/release/dist \ - --exclude "usr/src/release/*.txz" \ - --exclude "usr/src/release/*.orig" usr/src | \ - ${XZ_CMD} > ${.OBJDIR}/src.txz +# Create an mtree spec with faked timestamps so we get a reproducible +# tar file. We do not use tar for this because its mtree generator +# appears to be buggy and exits with an memory allocation failure. +# +# It's important that the excluded paths start with "./", otherwise +# file locations are not recorded correctly, and, for example, +# usr/src/usr.sbin appears as usr/src/release/usr.sbin in the +# tar file. Only the shadow knows if that's a bug or a feature. + echo "./usr/src/.git" >${.OBJDIR}/mtree-exclude + echo "./usr/src/release/dist" >>${.OBJDIR}/mtree-exclude + echo "./usr/src/release/src.mtree" >>${.OBJDIR}/mtree-exclude + echo "./usr/src/release/mtree-exclude" >>${.OBJDIR}/mtree-exclude + cd ${DISTDIR} && mtree -c -L -k time -X ${.OBJDIR}/mtree-exclude | \ + mtree -C | \ + sed -E -e 's@time=[0-9]+\.[0-9]+@time=${EPOCH_DATE}.0 uid=0 gid=0@' \ + -e 's@(type=dir)@\1 mode=0555@' \ + -e 's@(type=file)@\1 mode=0444@' \ + -e 's@//@/@g' \ + > ${.OBJDIR}/src.mtree + rm ${.OBJDIR}/mtree-exclude + cd ${DISTDIR} && tar cLvf - @${.OBJDIR}/src.mtree \ + | ${XZ_CMD} > ${.OBJDIR}/src.txz && \ + rm ${.OBJDIR}/src.mtree ports.txz: mkdir -p ${DISTDIR}/usr @@ -246,9 +264,6 @@ mini-memstick.img: bootonly sh ${.CURDIR}/${TARGET}/make-memstick.sh bootonly ${.TARGET} packagesystem: base.txz kernel.txz ${EXTRA_PACKAGES} - for tarball in ${EXTRA_PACKAGES}; do \ - sh ${.CURDIR}/scripts/tar-time-reset.sh $${tarball}; \ - done sh ${.CURDIR}/scripts/make-manifest.sh *.txz > MANIFEST touch ${.TARGET} diff --git a/release/scripts/tar-time-reset.sh b/release/scripts/tar-time-reset.sh index 8186fd8d3e26..f1b84340286d 100755 --- a/release/scripts/tar-time-reset.sh +++ b/release/scripts/tar-time-reset.sh @@ -17,11 +17,9 @@ ########################################################################## # # This script resets the timestamps in a given tarfile to hopefully make -# it reproducible. This is a rather wasteful approach, but works for now. -# -# A better solution would be to patch bsdtar to optionally use a fixed -# time (without having to clown around with mtree specs). -# +# it reproducible. As this is a rather wasteful approach the script is +# no longer used. It hasn't been removed yet as it may be useful for +# testing purposes. ########################################################################## main() { -- 2.32.0 From 1eb7f78e255d489a059d027fda44960b1bc84d10 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 25 May 2015 10:27:48 +0200 Subject: [PATCH 059/325] Add image-checksum.sh ... which calculates a checksum of the reproducible parts of an memstick image. Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 156 ++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100755 release/scripts/image-checksum.sh diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh new file mode 100755 index 000000000000..6f79a81ae37a --- /dev/null +++ b/release/scripts/image-checksum.sh @@ -0,0 +1,156 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## +# +# image-checksum.sh /path/to/memstick.img +# +# Unfortunately the memstick target currently does not create reproducible +# ElectroBSD images due to unreproducible differences in the file system +# layer of the data partition. +# +# To be able to (sort of) compare memstick images anyway, this script +# produces a "partial image checksum" that is based on the partition layout, +# the checksum of the boot code partition and an mtree spec of the data +# partition which includes checksums, sizes and timestamps for all the +# files. +# +# A memstick image whose "partial checksum" matches the one of another +# image can be totally considered to be nearly as trustworthy. Obviously +# that's a somewhat worthless property, it is thus recommended that you +# rebuild the potentionally malicious image using a trusted operating +# system first. After you've done this, potentionally malicious differences +# in the unchecked parts should be gone. +# +# Just kidding, image-checksum.sh is only intended to regression-test +# the ElectroBSD build system. +# +# Also note that this script relies on non-standardized output of other +# tools which might occasionally change. To be able to reproduce partial +# image checksums you thus need a userland that is close enough to the +# one that was used to create the original version. +# +########################################################################## + +UFS_PARTITION=p2 +EXPECTED_PARTITIONS=2 +MOUNTPOINT=/mnt +VERBOSE=0 +MTREE_KEYWORDS=size,time,uid,gid,sha256 + +verbose_log() { + local message="$*" + if [ "${VERBOSE}" = 0 ]; then + return + fi + echo "${message}" +} + +create_mtree_spec_file() { + local md_unit spec_file + + md_unit="${1}" + spec_file="${2}" + + verbose_log "Mounting /dev/md${md_unit}${UFS_PARTITION} at ${MOUNTPOINT}" + mount -o ro "/dev/md${md_unit}${UFS_PARTITION}" "${MOUNTPOINT}" || return 1 + + verbose_log "Running mtree, saving spec in ${spec_file}" + mtree -c -k "${MTREE_KEYWORDS}" -p "${MOUNTPOINT}" | mtree -C -k all > "${spec_file}" || return 1 + + verbose_log "Unmounting ${MOUNTPOINT} ..." + umount "${MOUNTPOINT}" || return 1 +} + +partition_count_acceptable() { + local md_unit="${1}" + + # Verify that there are exactly two partitions present + partitions=$(gpart show -r -p "md${md_unit}" | grep -c "md${md_unit}"p) + if [ "${partitions}" != "${EXPECTED_PARTITIONS}" ]; then + echo "Invalid number of partitions: ${partitions}" + return 1; + fi +} + +main() { + local image_file \ + args md_unit spec_file gpart_file + + args=$(getopt v $*) + if [ $? -ne 0 ]; then + echo 'You are doing it wrong: Invalid flag specified' + exit 2 + fi + set -- ${args} + while true; do + case "$1" in + -v) + VERBOSE=1 + shift + ;; + --) + shift; break + ;; + esac + done + + image_file=${1} + if [ -z "${image_file}" ]; then + echo "No image file provided" + return 1 + fi + spec_file="${image_file}.mtree" + if [ -f "${spec_file}" ]; then + echo "Spec file ${spec_file} already exists" + return 1 + fi + gpart_file="${image_file}.gpart" + if [ -f "${spec_file}" ]; then + echo "gpart file ${gpart_file} already exists" + return 1 + fi + + md_unit=$(mdconfig -o readonly -n -f "${image_file}") + if [ $? != 0 ]; then + return 1 + fi + + partition_count_acceptable "${md_unit}" || return 1 + + if [ ! -f "${spec_file}" ]; then + create_mtree_spec_file "${md_unit}" "${spec_file}" || return 1 + fi + if [ ! -f "${gpart_file}" ]; then + gpart list "md${md_unit}" | sed -E -e "s@(: md)${md_unit}@\1X@" > "${gpart_file}" + fi + + gpart_checksum=$(sha256 -q "${gpart_file}") + verbose_log "gpart checksum: ${gpart_checksum}" + + mdconfig -d -u "${md_unit}" || return 1 + + bootcode_checksum=$(dd if=/dev/md${md_unit}p1 2>/dev/null | sha256) + verbose_log "Boot code checksum: ${bootcode_checksum}" + mtree_checksum=$(sha256 -q "${spec_file}") + verbose_log "mtree checksum: ${mtree_checksum}" + + weak_image_checksum=$(echo "${gpart_checksum} ${bootcode_checksum} ${mtree_checksum}" | sha256) + echo "Partial image checksum: ${weak_image_checksum}" + +} + +main "${@}" -- 2.32.0 From fe4b8e5cc36de5bb6034ee1802cc1767ebe46f31 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 3 Jul 2015 19:54:10 +0200 Subject: [PATCH 060/325] image-checksum: Add -r flag to reuse cache files Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 6f79a81ae37a..1940921b44a1 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -50,6 +50,7 @@ EXPECTED_PARTITIONS=2 MOUNTPOINT=/mnt VERBOSE=0 MTREE_KEYWORDS=size,time,uid,gid,sha256 +REUSE_EXISTING_CACHE_FILES=false verbose_log() { local message="$*" @@ -90,7 +91,7 @@ main() { local image_file \ args md_unit spec_file gpart_file - args=$(getopt v $*) + args=$(getopt rv $*) if [ $? -ne 0 ]; then echo 'You are doing it wrong: Invalid flag specified' exit 2 @@ -98,6 +99,10 @@ main() { set -- ${args} while true; do case "$1" in + -r) + REUSE_EXISTING_CACHE_FILES=true + shift + ;; -v) VERBOSE=1 shift @@ -116,12 +121,12 @@ main() { spec_file="${image_file}.mtree" if [ -f "${spec_file}" ]; then echo "Spec file ${spec_file} already exists" - return 1 + ${REUSE_EXISTING_CACHE_FILES} || return 1 fi gpart_file="${image_file}.gpart" if [ -f "${spec_file}" ]; then echo "gpart file ${gpart_file} already exists" - return 1 + ${REUSE_EXISTING_CACHE_FILES} || return 1 fi md_unit=$(mdconfig -o readonly -n -f "${image_file}") -- 2.32.0 From 74c14fa35f7dcccc1b08cbbc4a612d93ed9054ba Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 3 Jul 2015 20:20:01 +0200 Subject: [PATCH 061/325] image-checksum.sh: Allow to overwrite the mtree flags Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 1940921b44a1..cc31e9055fe3 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -91,7 +91,7 @@ main() { local image_file \ args md_unit spec_file gpart_file - args=$(getopt rv $*) + args=$(getopt m:rv $*) if [ $? -ne 0 ]; then echo 'You are doing it wrong: Invalid flag specified' exit 2 @@ -99,6 +99,11 @@ main() { set -- ${args} while true; do case "$1" in + -m) + shift + MTREE_KEYWORDS="${1}" + shift + ;; -r) REUSE_EXISTING_CACHE_FILES=true shift -- 2.32.0 From 2590a7c8ad4d9ca00a8e32a52cd570b1de36771f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 31 Aug 2015 10:51:16 +0200 Subject: [PATCH 062/325] release/scripts/image-checksum.sh: Allow to checksum multiple images at once Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 79 +++++++++++++++++-------------- 1 file changed, 44 insertions(+), 35 deletions(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index cc31e9055fe3..30ac62a6c168 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -87,42 +87,11 @@ partition_count_acceptable() { fi } -main() { +generate_partial_image_checksum() { local image_file \ - args md_unit spec_file gpart_file - - args=$(getopt m:rv $*) - if [ $? -ne 0 ]; then - echo 'You are doing it wrong: Invalid flag specified' - exit 2 - fi - set -- ${args} - while true; do - case "$1" in - -m) - shift - MTREE_KEYWORDS="${1}" - shift - ;; - -r) - REUSE_EXISTING_CACHE_FILES=true - shift - ;; - -v) - VERBOSE=1 - shift - ;; - --) - shift; break - ;; - esac - done + md_unit spec_file gpart_file - image_file=${1} - if [ -z "${image_file}" ]; then - echo "No image file provided" - return 1 - fi + image_file="${1}" spec_file="${image_file}.mtree" if [ -f "${spec_file}" ]; then echo "Spec file ${spec_file} already exists" @@ -159,8 +128,48 @@ main() { verbose_log "mtree checksum: ${mtree_checksum}" weak_image_checksum=$(echo "${gpart_checksum} ${bootcode_checksum} ${mtree_checksum}" | sha256) - echo "Partial image checksum: ${weak_image_checksum}" + echo "Partial image checksum for ${image_file}: ${weak_image_checksum}" +} +main() { + local image_file \ + args + + args=$(getopt m:rv $*) + if [ $? -ne 0 ]; then + echo 'You are doing it wrong: Invalid flag specified' + exit 2 + fi + set -- ${args} + while true; do + case "$1" in + -m) + shift + MTREE_KEYWORDS="${1}" + shift + ;; + -r) + REUSE_EXISTING_CACHE_FILES=true + shift + ;; + -v) + VERBOSE=1 + shift + ;; + --) + shift; break + ;; + esac + done + + if [ -z "${1}" ]; then + echo "No image file provided" + return 1 + fi + + for image_file in "${@}"; do + generate_partial_image_checksum "${image_file}" || return 1 + done } main "${@}" -- 2.32.0 From fcd792f877dc7adb2a47f9969e62f4fd7d1b8c83 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 30 May 2015 14:38:48 +0200 Subject: [PATCH 063/325] sys: Do not embed the compiler version in the kernel binary ... as it makes reproducing the binary with a different compiler more complicated. In case of ElectroBSD the compiler used can be usually deduced from the uname output anyway as the upstream revision is part of the fake hostname when using reproduce.sh. Obtained from: ElectroBSD --- sys/conf/newvers.sh | 2 -- sys/kern/init_main.c | 1 - sys/kern/kern_mib.c | 3 --- sys/sys/systm.h | 1 - 4 files changed, 7 deletions(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index 11025ba4cde2..782021850741 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -147,7 +147,6 @@ else t=`date` fi i=`${MAKE:-make} -V KERN_IDENT` -compiler_v=$($(${MAKE:-make} -V CC) -v 2>&1 | grep -w 'version') for dir in /usr/bin /usr/local/bin; do if [ ! -z "${svnversion}" ] ; then @@ -293,7 +292,6 @@ $COPYRIGHT char sccs[sizeof(SCCSSTR) > 128 ? sizeof(SCCSSTR) : 128] = SCCSSTR; char version[sizeof(VERSTR) > 256 ? sizeof(VERSTR) : 256] = VERSTR; -char compiler_version[] = "${compiler_v}"; char ostype[] = "${TYPE}"; char osrelease[sizeof(RELSTR) > 32 ? sizeof(RELSTR) : 32] = RELSTR; int osreldate = ${RELDATE}; diff --git a/sys/kern/init_main.c b/sys/kern/init_main.c index 06f702c1c34b..c90558eb5436 100644 --- a/sys/kern/init_main.c +++ b/sys/kern/init_main.c @@ -355,7 +355,6 @@ print_version(void *data __unused) while (len > 0 && version[len - 1] == '\n') len--; printf("%.*s %s\n", len, version, machine); - printf("%s\n", compiler_version); } SYSINIT(announce, SI_SUB_COPYRIGHT, SI_ORDER_FIRST, print_caddr_t, diff --git a/sys/kern/kern_mib.c b/sys/kern/kern_mib.c index ed44645246d4..ef4891955279 100644 --- a/sys/kern/kern_mib.c +++ b/sys/kern/kern_mib.c @@ -103,9 +103,6 @@ SYSCTL_INT(_kern, KERN_OSREV, osrevision, CTLFLAG_RD|CTLFLAG_CAPRD, SYSCTL_STRING(_kern, KERN_VERSION, version, CTLFLAG_RD|CTLFLAG_MPSAFE, version, 0, "Kernel version"); -SYSCTL_STRING(_kern, OID_AUTO, compiler_version, CTLFLAG_RD|CTLFLAG_MPSAFE, - compiler_version, 0, "Version of compiler used to compile kernel"); - SYSCTL_STRING(_kern, KERN_OSTYPE, ostype, CTLFLAG_RD|CTLFLAG_MPSAFE| CTLFLAG_CAPRD, ostype, 0, "Operating system type"); diff --git a/sys/sys/systm.h b/sys/sys/systm.h index 0c1d392fd740..5c7bf9939328 100644 --- a/sys/sys/systm.h +++ b/sys/sys/systm.h @@ -54,7 +54,6 @@ extern int suspend_blocked; /* block suspend due to pending shutdown */ extern int rebooting; /* kern_reboot() has been called. */ extern const char *panicstr; /* panic message */ extern char version[]; /* system version */ -extern char compiler_version[]; /* compiler version */ extern char copyright[]; /* system copyright */ extern int kstack_pages; /* number of kernel stack pages */ -- 2.32.0 From 3a172dd814defd9d7f6471529a525288edbd78d0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Jun 2015 12:18:16 +0200 Subject: [PATCH 064/325] release/Makefile: Reorder dependencies to reduce the chances that base or kernel parts end up in the src.txz. This is merely a workaround Obtained from: ElectroBSD --- release/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/Makefile b/release/Makefile index d90074ada1cc..2d7198019c1a 100644 --- a/release/Makefile +++ b/release/Makefile @@ -263,7 +263,7 @@ mini-memstick: mini-memstick.img mini-memstick.img: bootonly sh ${.CURDIR}/${TARGET}/make-memstick.sh bootonly ${.TARGET} -packagesystem: base.txz kernel.txz ${EXTRA_PACKAGES} +packagesystem: ${EXTRA_PACKAGES} base.txz kernel.txz sh ${.CURDIR}/scripts/make-manifest.sh *.txz > MANIFEST touch ${.TARGET} -- 2.32.0 From c001722514797db7610e60381f62d34722ce04f3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 17:30:25 +0200 Subject: [PATCH 065/325] Add strip-freebsd.sh ... which suggests a bunch of stuff to delete from a vanilla FreeBSD checkout. In a previous life it was called free-freebsd.sh which obviously sounds more awesome, but nowadays most of the suggested stuff for removal is actually free software that just isn't relevant for ElectroBSD. Removing code we don't need means we don't have to care about its security and license issues. There's lots of code to remove left! Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 183 +++++++++++++++++++++++++++++++ 1 file changed, 183 insertions(+) create mode 100755 release/scripts/strip-freebsd.sh diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh new file mode 100755 index 000000000000..e805f9cab8e4 --- /dev/null +++ b/release/scripts/strip-freebsd.sh @@ -0,0 +1,183 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## + +# This script pretends to free a FreeBSD checkout from +# known-unfree files and other stuff that is not required +# by ElectroBSD. +# +# While it is pretty much guaranteed to ditch a bunch of files +# it doesn't work very thorougly and the resulting checkout is +# likely to still contain lots of non-free parts that haven't +# been discovered yet. + +get_snd_csa_files() { + find sys/dev/sound/pci -name "csa*" +} + +get_snd_ds1_files() { + find sys/dev/sound/pci -name "ds1*" +} + +get_snd_maestro3_files() { + # This file contains the actual blobs + echo sys/dev/sound/pci/allegro_code.h + + # This file contains the (free) code that relies on the blobs. + echo sys/dev/sound/pci/maestro3.c +} + +# We can't simply remove the whole directory +# as a (free) header is required by bge. +get_bce_files() { + echo "sys/dev/bce/if_bcefw.h" \ + "sys/dev/bce/if_bce.c" +} + +get_usb_firmware_files() { + find sys/dev/usb/ -name "*fw*" +} + +# XXX: Misleading name, some of the files are merely tainted +# by non-free dependencies +get_unfree_files() { + find sys/ -name "*.uu" + get_snd_csa_files + get_snd_ds1_files + get_snd_maestro3_files + get_bce_files + get_usb_firmware_files +} + +get_files_to_ditch() { + get_unfree_files +} + +# These architectures are mainly unsupported by ElectroBSD +# due to lack of hardware for testing purposes. +# +# The source directories are mainly removed to shrink the +# source tarball and to reduce the number of files that +# should be audited for license and security issues. +get_unsupported_architectures() { + echo "arm arm64 mips pc98 powerpc riscv sparc64" +} + +# These depend on or contain proprietary firmware that is included in sys/contrib/dev +get_tainted_sys_contrib_devs() { + echo "drm2 ipw iwi iwm iwn mwl npe otus ral rsu rtwn run uath urtwn wpi" +} + +# These require proprietary firmware that is included in sys/dev +# and may cause build failures without it. +get_tainted_sys_devs() { + # bce has already been taken care of by get_bce_files() above + echo "bxe ctau cx cxgb cxgbe ispfw qlxgbe" \ + "it tw" +} + +get_unused_contrib_dirs() { + # XXX: gcc can't be deleted because parts of it are apparently + # required to build libc. This should be investigated more thoroughly, + # hopefully it can be fixed. + echo "apr apr-util ipfilter ofed sendmail serf subversion tcsh" +} + +# Only includes directory that aren't architecture-specific +get_unused_cross_platform_sys_dirs() { + echo "netnatm ofed" +} + +get_directories_to_ditch() { + local arch \ + dir arch_dir sys_contrib contrib_dir + + for dir in sys sys/boot; do + for arch in $(get_unsupported_architectures); do + potential_directory="${dir}/${arch}" + if [ -d "${potential_directory}" ]; then + echo "${potential_directory}" + fi + done + done + + for dir in $(get_unused_cross_platform_sys_dirs); do + potential_directory="sys/${dir}" + if [ -d "${potential_directory}" ]; then + echo "${potential_directory}" + fi + done + + for sys_contrib in ipfilter ncsw octeon-sdk; do + echo "sys/contrib/${sys_contrib}" + done + + for sys_contrib in $(get_tainted_sys_contrib_devs); do + echo "sys/contrib/dev/${sys_contrib}" + done + + for sys_dev in $(get_tainted_sys_devs); do + echo "sys/dev/${sys_dev}" + done + + for contrib_dir in $(get_unused_contrib_dirs); do + potential_directory="contrib/${contrib_dir}" + if [ -d "${potential_directory}" ]; then + echo "${potential_directory}" + fi + done +} + +purify_cwd() { + # There are no spaces in paths or file names. + files_to_ditch="$(get_files_to_ditch)" + for f in $files_to_ditch; do + [ -f "${f}" ] && echo "rm ${f}" + done + + dirs_to_ditch="$(get_directories_to_ditch)" + for d in $dirs_to_ditch; do + [ -d "${d}" ] && echo "rm -r ${d}" + done +} + +main() { + local src_dir \ + files_to_ditch dirs_to_ditch + + src_dir="${1}" + if [ -z "${src_dir}" ]; then + echo "No source directory given" + return 1 + fi + if [ ! -d "${src_dir}" ]; then + echo "No such directory: ${src_dir}" + return 1 + fi + # Make it less likely to operate on a directory + # that isn't actually a FreeBSD checkout + if [ ! -f "${src_dir}/COPYRIGHT" ]; then + echo "${src_dir} contains no COPYRIGHT file" + return 1 + fi + + cd "${src_dir}" || return 1 + + purify_cwd +} + +main "${@}" -- 2.32.0 From d89ebd62b788ceb2fc793b474b10e605662a1d95 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 7 Dec 2016 12:25:38 +0100 Subject: [PATCH 066/325] Detach telnetd from the build Obtained from: ElectroBSD --- secure/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/secure/Makefile b/secure/Makefile index 198df326fe32..29d7f7b13ef8 100644 --- a/secure/Makefile +++ b/secure/Makefile @@ -12,7 +12,7 @@ SUBDIR.${MK_CAROOT}+= caroot # These are the programs which depend on crypto, but not Kerberos. SPROGS= lib/libfetch lib/libpam lib/libradius lib/libtelnet \ - bin/ed libexec/telnetd usr.bin/fetch usr.bin/telnet \ + bin/ed usr.bin/fetch usr.bin/telnet \ usr.sbin/ppp usr.sbin/tcpdump/tcpdump .if ${MK_SENDMAIL} != "no" SPROGS+=usr.sbin/sendmail -- 2.32.0 From 5d54fcd2dfae2908c25969bcaad4449e85720a07 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 4 Nov 2015 18:55:18 +0100 Subject: [PATCH 067/325] share/doc: Detach 'legal' from the build It (tries to) install proprietary licenses for code that is not part of ElectroBSD and deleted by reproduce.sh's auto-untaint mode (-a). Obtained from: ElectroBSD --- share/doc/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/share/doc/Makefile b/share/doc/Makefile index 7718c6344dc4..80bdc6843034 100644 --- a/share/doc/Makefile +++ b/share/doc/Makefile @@ -5,7 +5,6 @@ SUBDIR= ${_IPv6} \ ${_atf} \ - legal \ ${_llvm} \ ${_pjdfstest} \ ${_roffdocs} -- 2.32.0 From 881216803e0f044c0d7874d8582849a4c9d5571a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 21 May 2015 18:42:42 +0200 Subject: [PATCH 068/325] release/Makefile: Export VOLUME_LABEL for make-memstick.sh Obtained from: ElectroBSD --- release/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/release/Makefile b/release/Makefile index 2d7198019c1a..930273c92e88 100644 --- a/release/Makefile +++ b/release/Makefile @@ -77,6 +77,8 @@ VOLUME_LABEL= ${REVISION:C/[.-]/_/g}_${BRANCH:C/[.-]/_/g}_${TARGET_ARCH} VOLUME_LABEL= FreeBSD_Install .endif +.export VOLUME_LABEL + .if !exists(${PORTSDIR}) || !defined(WITH_PORTS) NOPORTS= true .endif -- 2.32.0 From 7993cd84394ba98fd0d5c7602e6ed86165a59aee Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 3 Dec 2015 18:34:21 +0100 Subject: [PATCH 069/325] lib/libkvm: Unbreak the build without kernel sources that don't ship with ElectroBSD Obtained from: ElectroBSD --- lib/libkvm/Makefile | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/lib/libkvm/Makefile b/lib/libkvm/Makefile index d934690d4355..a814ec9a64ee 100644 --- a/lib/libkvm/Makefile +++ b/lib/libkvm/Makefile @@ -12,14 +12,8 @@ WARNS?= 6 SRCS= kvm.c kvm_cptime.c kvm_getloadavg.c \ kvm_getswapinfo.c kvm_pcpu.c kvm_private.c kvm_proc.c kvm_vnet.c \ - kvm_minidump_aarch64.c \ kvm_amd64.c kvm_minidump_amd64.c \ - kvm_arm.c kvm_minidump_arm.c \ - kvm_i386.c kvm_minidump_i386.c \ - kvm_minidump_mips.c \ - kvm_powerpc.c kvm_powerpc64.c \ - kvm_minidump_riscv.c \ - kvm_sparc64.c + kvm_i386.c kvm_minidump_i386.c INCS= kvm.h LIBADD= elf -- 2.32.0 From 81e802dea20a63787234d1b59ef02d7ae6c04495 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 24 May 2015 14:03:06 +0200 Subject: [PATCH 070/325] usr.sbin/pkg: Change URL_SCHEME_PREFIX to an URL more easily controlled by the user ... through ssh port forwarding. Obtained from: ElectroBSD --- usr.sbin/pkg/config.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/usr.sbin/pkg/config.c b/usr.sbin/pkg/config.c index f29bd586f338..a4a94ac8c742 100644 --- a/usr.sbin/pkg/config.c +++ b/usr.sbin/pkg/config.c @@ -64,7 +64,7 @@ static struct config_entry c[] = { [PACKAGESITE] = { PKG_CONFIG_STRING, "PACKAGESITE", - URL_SCHEME_PREFIX "http://pkg.FreeBSD.org/${ABI}/latest", + URL_SCHEME_PREFIX "http://127.0.0.1:8000/packages/${ABI}/", NULL, NULL, false, -- 2.32.0 From 3b9e617eb6160a271fb81bd0c01d09b8433f7ea9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 25 May 2015 15:33:59 +0200 Subject: [PATCH 071/325] share/doc: Exclude directories that contain (partly) non-free materials Obtained from: ElectroBSD --- share/doc/Makefile | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/share/doc/Makefile b/share/doc/Makefile index 80bdc6843034..dc2003f7ecfe 100644 --- a/share/doc/Makefile +++ b/share/doc/Makefile @@ -3,11 +3,9 @@ .include -SUBDIR= ${_IPv6} \ - ${_atf} \ +SUBDIR= ${_atf} \ ${_llvm} \ - ${_pjdfstest} \ - ${_roffdocs} + ${_pjdfstest} .if ${MK_TESTS} != "no" _atf= atf @@ -18,9 +16,11 @@ _pjdfstest= pjdfstest _llvm= llvm .endif -.if ${MK_INET6} != "no" -_IPv6= IPv6 -.endif +# Note: Documents in the sub directories IPv6, papers, psd, smm +# and usd are not installed on ElectroBSD due to license problems. +# +# Some of the excluded papers actually have free licenses and +# should eventually be installed. SUBDIR_PARALLEL= -- 2.32.0 From 37724191ac0c024abce09c0055accefab74ddf42 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 May 2015 10:10:43 +0200 Subject: [PATCH 072/325] geom: Do not build raid, raid3 and vinum classes Obtained from: ElectroBSD --- sys/modules/geom/Makefile | 3 --- 1 file changed, 3 deletions(-) diff --git a/sys/modules/geom/Makefile b/sys/modules/geom/Makefile index cbc7207b6f0e..25b99cd94cec 100644 --- a/sys/modules/geom/Makefile +++ b/sys/modules/geom/Makefile @@ -17,13 +17,10 @@ SUBDIR= geom_bde \ geom_multipath \ geom_nop \ geom_part \ - geom_raid \ - geom_raid3 \ geom_sched \ geom_shsec \ geom_stripe \ geom_uzip \ - geom_vinum \ geom_virstor \ geom_zero -- 2.32.0 From 90819a771e851018ac28b30148c20ed7fe904b38 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 May 2015 12:17:36 +0200 Subject: [PATCH 073/325] share/mk/src.opts.mk: Disable a bunch of options by default .. either due to license issues or because they are not considered relevant for the majority of ElectroBSD users. src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 65 ++++++++++++++++++++++++-------------------- 1 file changed, 36 insertions(+), 29 deletions(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 4fd16579f863..0052e86a7d07 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -58,7 +58,6 @@ __DEFAULT_YES_OPTIONS = \ AMD \ APM \ AT \ - ATM \ AUDIT \ AUTHPF \ AUTOFS \ @@ -66,24 +65,21 @@ __DEFAULT_YES_OPTIONS = \ BINUTILS \ BINUTILS_BOOTSTRAP \ BLACKLIST \ - BLUETOOTH \ BOOT \ BOOTPARAMD \ BOOTPD \ BSD_CPIO \ - BSDINSTALL \ + BSD_GREP_FASTMATCH \ BSNMP \ BZIP2 \ CALENDAR \ CAPSICUM \ CAROOT \ CASPER \ - CCD \ CDDL \ CPP \ CROSS_COMPILER \ CRYPT \ - CTM \ CUSE \ CXX \ DIALOG \ @@ -98,14 +94,9 @@ __DEFAULT_YES_OPTIONS = \ FDT \ FILE \ FINGER \ - FLOPPY \ - FMTREE \ FORTH \ FP_LIBC \ - FREEBSD_UPDATE \ - FTP \ GAMES \ - GCOV \ GDB \ GNU_DIFF \ GNU_GREP \ @@ -118,9 +109,7 @@ __DEFAULT_YES_OPTIONS = \ INET \ INET6 \ INETD \ - IPFILTER \ IPFW \ - ISCSI \ JAIL \ KDUMP \ KVM \ @@ -145,7 +134,7 @@ __DEFAULT_YES_OPTIONS = \ MAILWRAPPER \ MAKE \ MALLOC_PRODUCTION \ - NDIS \ + MANDOCDB \ NETCAT \ NETGRAPH \ NLS_CATALOGS \ @@ -154,36 +143,22 @@ __DEFAULT_YES_OPTIONS = \ OFED \ OPENSSL \ PAM \ - PC_SYSINSTALL \ PF \ PKGBOOTSTRAP \ - PMC \ - PORTSNAP \ - PPP \ QUOTAS \ RADIUS_SUPPORT \ - RBOOTD \ REPRODUCIBLE_BUILD \ RESCUE \ - ROUTED \ - SENDMAIL \ - SERVICESDB \ + RCS \ + RESCUE \ SETUID_LOGIN \ SHAREDOCS \ - SOURCELESS \ - SOURCELESS_HOST \ - SOURCELESS_UCODE \ - SVNLITE \ SYSCONS \ SYSTEM_COMPILER \ SYSTEM_LINKER \ TALK \ - TCP_WRAPPERS \ - TCSH \ TELNET \ TEXTPROC \ - TFTP \ - TIMED \ UNBOUND \ USB \ UTMPX \ @@ -245,6 +220,38 @@ __DEFAULT_DEPENDENT_OPTIONS= \ __DEFAULT_DEPENDENT_OPTIONS+= ${var}_SUPPORT/${var} .endfor +# Disable a bunch of additional options that default to yes in FreeBSD +__DEFAULT_NO_OPTIONS += \ + ATM \ + BLUETOOTH \ + BSDINSTALL \ + CCD \ + CTM \ + FLOPPY \ + FMTREE \ + FREEBSD_UPDATE \ + FTP \ + GCOV \ + IPFILTER \ + ISCSI \ + NDIS \ + RBOOTD \ + PC_SYSINSTALL \ + PMC \ + PORTSNAP \ + PPP \ + RCMDS \ + SENDMAIL \ + SVNLITE \ + SOURCELESS \ + SOURCELESS_HOST \ + SOURCELESS_UCODE \ + SYSINSTALL \ + TCP_WRAPPERS \ + TCSH \ + TFTP \ + TIMED \ + # # Default behaviour of some options depends on the architecture. Unfortunately # this means that we have to test TARGET_ARCH (the buildworld case) as well -- 2.32.0 From 443f39dc4c1a17296afdc7d7000518613a3c7766 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 24 Oct 2015 12:58:42 +0200 Subject: [PATCH 074/325] share/mk/src.opts.mk: Disable EXAMPLES ... so we don't have to care about license issues. Some of the examples are non-free, in some cases the license is unclear. XXX: Some of the examples are properly licensed and moderately useful, for example share/examples/bhyve/vmrun.sh. It can be executed from a checkout directly, though. src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 0052e86a7d07..fcf5ba638533 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -90,7 +90,6 @@ __DEFAULT_YES_OPTIONS = \ EE \ EFI \ ELFTOOLCHAIN_BOOTSTRAP \ - EXAMPLES \ FDT \ FILE \ FINGER \ @@ -227,6 +226,7 @@ __DEFAULT_NO_OPTIONS += \ BSDINSTALL \ CCD \ CTM \ + EXAMPLES \ FLOPPY \ FMTREE \ FREEBSD_UPDATE \ -- 2.32.0 From d0450619a294658085851645946ed66285600cb6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 24 Jun 2015 12:53:20 +0200 Subject: [PATCH 075/325] share/mk/src.opts.mk: Enable CLANG_EXTRAS by default as it's required for llvm-symbolizer src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index fcf5ba638533..87304b8b5871 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -173,7 +173,6 @@ __DEFAULT_NO_OPTIONS = \ BEARSSL \ BSD_CRTBEGIN \ BSD_GREP \ - CLANG_EXTRAS \ CLANG_FORMAT \ DTRACE_TESTS \ GH_BC \ @@ -219,6 +218,10 @@ __DEFAULT_DEPENDENT_OPTIONS= \ __DEFAULT_DEPENDENT_OPTIONS+= ${var}_SUPPORT/${var} .endfor +# Enable additional options that default to NO in FreeBSD +__DEFAULT_YES_OPTIONS += \ + CLANG_EXTRAS \ + # Disable a bunch of additional options that default to yes in FreeBSD __DEFAULT_NO_OPTIONS += \ ATM \ -- 2.32.0 From 1310aa8c458bcc8b2eec65f2b48c4497ca8fd3a6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 May 2015 17:53:19 +0200 Subject: [PATCH 076/325] etc/master.passwd: Change root's login shell to sh Now that csh is no longer compiled by default that seems like a rather swell idea. Obtained from: ElectroBSD --- etc/master.passwd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/etc/master.passwd b/etc/master.passwd index a1be886f1e8d..b61c13fdd9fe 100644 --- a/etc/master.passwd +++ b/etc/master.passwd @@ -1,6 +1,6 @@ # $FreeBSD$ # -root::0:0::0:0:Charlie &:/root:/bin/csh +root::0:0::0:0:Charlie &:/root:/bin/sh toor:*:0:0::0:0:Bourne-again Superuser:/root: daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin operator:*:2:5::0:0:System &:/:/usr/sbin/nologin -- 2.32.0 From c0fdb10ac42ea4304307ca96ea459cee465faf82 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 5 Jun 2015 12:03:46 +0200 Subject: [PATCH 077/325] release/Makefile: Set German keyboard map Obtained from: ElectroBSD --- release/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/release/Makefile b/release/Makefile index 930273c92e88..e531e0e86dc8 100644 --- a/release/Makefile +++ b/release/Makefile @@ -196,6 +196,7 @@ disc1: packagesystem ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf echo sendmail_enable=\"NONE\" > ${.TARGET}/etc/rc.conf echo hostid_enable=\"NO\" >> ${.TARGET}/etc/rc.conf + echo keymap=\"de\" >> ${.TARGET}/etc/rc.conf echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf cp ${.CURDIR}/rc.local ${.TARGET}/etc -- 2.32.0 From 582b5b3593b76375fafd4106e2167c0e94b0095a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 5 Jun 2015 12:06:14 +0200 Subject: [PATCH 078/325] release/Makefile: Set hostname to ${VOLUME_LABEL} Obtained from: ElectroBSD --- release/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/release/Makefile b/release/Makefile index e531e0e86dc8..d68e4edbcef3 100644 --- a/release/Makefile +++ b/release/Makefile @@ -197,6 +197,7 @@ disc1: packagesystem echo sendmail_enable=\"NONE\" > ${.TARGET}/etc/rc.conf echo hostid_enable=\"NO\" >> ${.TARGET}/etc/rc.conf echo keymap=\"de\" >> ${.TARGET}/etc/rc.conf + echo hostname=\"${VOLUME_LABEL}\" >> ${.TARGET}/etc/rc.conf echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf cp ${.CURDIR}/rc.local ${.TARGET}/etc -- 2.32.0 From 304a7b616167e3b291c437273ea28a60a2a12a55 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 15 Jan 2017 11:38:02 +0100 Subject: [PATCH 079/325] release/Makefile: Use the OSRELEASE verbatim as VOLUME_LABEL ... but shorten if the label would be too long otherwise. On ElectroBSD the OSRELEASE only contains useful information. Obtained from: ElectroBSD --- release/Makefile | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/release/Makefile b/release/Makefile index d68e4edbcef3..9a8891b4c221 100644 --- a/release/Makefile +++ b/release/Makefile @@ -65,11 +65,13 @@ ${_V}!= eval $$(awk '/^${_V}=/{print}' ${.CURDIR}/../sys/conf/newvers.sh); echo .for _V in ${TARGET_ARCH} .if !empty(TARGET:M${_V}) OSRELEASE= ${TYPE}-${REVISION}-${BRANCH}-${TARGET} -VOLUME_LABEL= ${REVISION:C/[.-]/_/g}_${BRANCH:C/[.-]/_/g}_${TARGET} .else OSRELEASE= ${TYPE}-${REVISION}-${BRANCH}-${TARGET}-${TARGET_ARCH} -VOLUME_LABEL= ${REVISION:C/[.-]/_/g}_${BRANCH:C/[.-]/_/g}_${TARGET_ARCH} .endif +# The label length needs to be below 32 characters to fit into +# the UFS superblock. PRERELEASE is the only branch name that +# can prevent this, so we shorten it. +VOLUME_LABEL= ${OSRELEASE:C/PRERELEASE/PREREL/} .endfor .endif -- 2.32.0 From 691d5af0c2cfd426840c921be7d93dc075c72f4f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 5 Jun 2015 19:38:39 +0200 Subject: [PATCH 080/325] Stop bothering packaging docs which are no longer built. Squash or relocate Obtained from: ElectroBSD --- Makefile.inc1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.inc1 b/Makefile.inc1 index 74426f0984dd..48c66e9e0e40 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1355,7 +1355,7 @@ ITOOLS+=makewhatis # # Non-base distributions produced by the base system -EXTRA_DISTRIBUTIONS= doc +EXTRA_DISTRIBUTIONS= .if defined(LIBCOMPAT) EXTRA_DISTRIBUTIONS+= lib${libcompat} .endif -- 2.32.0 From 665dd53cd5d426ae00c9c3fb52eafcdb6f0a2061 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 10 Jun 2015 12:29:37 +0200 Subject: [PATCH 081/325] release/Makefile: Add ${DIST_TARBALL_DIR} ... which contains the distribution tarball directory on the install media. Obtained from: ElectroBSD --- release/Makefile | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/release/Makefile b/release/Makefile index 9a8891b4c221..0473132c14da 100644 --- a/release/Makefile +++ b/release/Makefile @@ -57,6 +57,9 @@ DISTDIR= dist NO_ROOT=1 .export NO_ROOT +# Path to the distribution tarballs on the created installation media. +DIST_TARBALL_DIR=usr/freebsd-dist + # Define OSRELEASE by using newvars.sh .if !defined(OSRELEASE) || empty(OSRELEASE) .for _V in TYPE BRANCH REVISION @@ -190,9 +193,9 @@ disc1: packagesystem MK_KERNEL_SYMBOLS=no MK_TESTS=no MK_DEBUG_FILES=no \ -DDB_FROM_SRC # Copy distfiles - mkdir -p ${.TARGET}/usr/freebsd-dist + mkdir -p ${.TARGET}/${DIST_TARBALL_DIR} for dist in MANIFEST $$(ls *.txz | grep -vE -- '(base|lib32)-dbg'); \ - do cp $${dist} ${.TARGET}/usr/freebsd-dist; \ + do cp $${dist} ${.TARGET}/${DIST_TARBALL_DIR}; \ done # Set up installation environment ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf @@ -217,8 +220,9 @@ bootonly: packagesystem MK_KERNEL_SYMBOLS=no MK_TESTS=no MK_DEBUG_FILES=no \ -DDB_FROM_SRC # Copy manifest only (no distfiles) to get checksums - mkdir -p ${.TARGET}/usr/freebsd-dist - cp MANIFEST ${.TARGET}/usr/freebsd-dist + mkdir -p ${.TARGET}/${DIST_TARBALL_DIR} + cp MANIFEST ${.TARGET}/${DIST_TARBALL_DIR} + # Set up installation environment ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf echo sendmail_enable=\"NONE\" > ${.TARGET}/etc/rc.conf @@ -235,9 +239,9 @@ dvd: packagesystem MK_TESTS=no MK_DEBUG_FILES=no \ -DDB_FROM_SRC # Copy distfiles - mkdir -p ${.TARGET}/usr/freebsd-dist + mkdir -p ${.TARGET}/${DIST_TARBALL_DIR} for dist in MANIFEST $$(ls *.txz | grep -v -- '(base|lib32)-dbg'); \ - do cp $${dist} ${.TARGET}/usr/freebsd-dist; \ + do cp $${dist} ${.TARGET}/${DIST_TARBALL_DIR}; \ done # Set up installation environment ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf -- 2.32.0 From 5e92b981181cef020b9a07165807c4630de5e81d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 10 Jun 2015 12:36:37 +0200 Subject: [PATCH 082/325] release/Makefile: ElectroBSDify the shiny new ${DIST_TARBALL_DIR} Obtained from: ElectroBSD --- release/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/Makefile b/release/Makefile index 0473132c14da..7f46e08a42cd 100644 --- a/release/Makefile +++ b/release/Makefile @@ -58,7 +58,7 @@ NO_ROOT=1 .export NO_ROOT # Path to the distribution tarballs on the created installation media. -DIST_TARBALL_DIR=usr/freebsd-dist +DIST_TARBALL_DIR=usr/electrobsd-dist # Define OSRELEASE by using newvars.sh .if !defined(OSRELEASE) || empty(OSRELEASE) -- 2.32.0 From 9e467bf5dff2f6d73cd0b5b95e3b8cc90ba37702 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 15 Jun 2015 12:57:51 +0200 Subject: [PATCH 083/325] Add rc.d script to automatically enable soft-protection on boot Obtained from: ElectroBSD --- etc/rc.d/soft-protection | 46 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100755 etc/rc.d/soft-protection diff --git a/etc/rc.d/soft-protection b/etc/rc.d/soft-protection new file mode 100755 index 000000000000..58bc02a0c037 --- /dev/null +++ b/etc/rc.d/soft-protection @@ -0,0 +1,46 @@ +#!/bin/sh +# +########################################################################### +# +# soft-protection - Enables cloudiatr soft protection on boot +# +########################################################################### +# +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ALL YOUR +# DATA IS BELONG TO THE SOFTWARE AND MAY BE EATEN BY IT. IF THAT IS NOT +# ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. +########################################################################### + +# PROVIDE: soft_protection +# REQUIRE: FILESYSTEMS + +. /etc/rc.subr + +name="soft_protection" +rcvar="soft_protection_enable" + +soft_protection_enable="${soft_protection_enable-NO}" + +start_cmd="enable_soft_protection" +stop_cmd=":" + +enable_soft_protection() { + # We don't use the soft-protect subcommand + # because it may require user feedback. + cloudiatr cmd cloudiatr_soft_protect +} + +load_rc_config "${name}" +run_rc_command "${1}" -- 2.32.0 From 7f516c587e0eefe06aaea1730720ec66cf18032a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 23 Jul 2015 19:16:44 +0200 Subject: [PATCH 084/325] dtrace_consume(): Warn if cpu cores aren't completely in sync ... instead of aborting. While the results may be not completely accurate, in some cases it may not matter. Previously the assertion would sometimes be triggered on a systems with poor timecounters (TSC-low(-100) ACPI-fast(900) i8254(0) dummy(-1000000)). Obtained from: ElectroBSD --- .../opensolaris/lib/libdtrace/common/dt_consume.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c index 51d7f332a28d..80db09254dd0 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c @@ -30,6 +30,7 @@ #include #include +#include #include #include #include @@ -3023,7 +3024,13 @@ dtrace_consume(dtrace_hdl_t *dtp, FILE *fp, if (buf != NULL) { if (first_timestamp == 0) first_timestamp = buf->dtbd_timestamp; - assert(buf->dtbd_timestamp >= first_timestamp); + if (buf->dtbd_timestamp < first_timestamp) { + warnx("cpu clocks out of sync " + "(%ju < %ju; offset: %ju). " + "Results may be incorrect!", + buf->dtbd_timestamp, first_timestamp, + first_timestamp - buf->dtbd_timestamp); + } dt_pq_insert(dtp->dt_bufq, buf); drops[i] = buf->dtbd_drops; -- 2.32.0 From b423843f566173c0a3913ecc850a24c6a915c774 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 13 Oct 2015 18:19:42 +0200 Subject: [PATCH 085/325] libdtrace: Replace another timestamp related assert() with a warning It triggered a lot less often, but was still annoying. Example: 2015 Oct 13 18:01:06: v_free_target: 20888, v_free_count: 44001, v_cache_count: 0. 2015 Oct 13 18:01:06: v_inactive_target: 31332. v_inactive_count: 364311. deficit: 0 2015 Oct 13 18:01:06: after-inactive-scan pass 3: page shortage: -23113, inactive page surplus: 10000, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:06: before-active-scan pass 3: page_shortage: -356092. 2015 Oct 13 18:01:06: after-active-scan pass 3: page_shortage: -356092. 2015 Oct 13 18:01:06: checked-inactive-pages: Inactive page surplus: 10000, above limit: 252979. dtrace: cpu clocks out of sync (18128465664478 < 18128471798881; offset: 6134403). Results may be incorrect! dtrace: cpu clocks out of sync (18129463671844 < 18129469778266; offset: 6106422). Results may be incorrect! 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253446. 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253446. 2015 Oct 13 18:01:07: before-inactive-scan: Scan goal 1: Move inactive to cache or free. 2015 Oct 13 18:01:07: before-inactive-scan: Page shortage: -37096, inactive page surplus: 10000 2015 Oct 13 18:01:07: v_free_target: 20888, v_free_count: 57984, v_cache_count: 0. 2015 Oct 13 18:01:07: v_inactive_target: 31332. v_inactive_count: 364778. deficit: 0 2015 Oct 13 18:01:07: after-inactive-scan pass 1: page shortage: -37203, inactive page surplus: 9893, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253250. 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253250. 2015 Oct 13 18:01:07: before-inactive-scan: Scan goal 2: Launder dirty pages. 2015 Oct 13 18:01:07: before-inactive-scan: Page shortage: -37191, inactive page surplus: 10000 2015 Oct 13 18:01:07: v_free_target: 20888, v_free_count: 58079, v_cache_count: 0. 2015 Oct 13 18:01:07: v_inactive_target: 31332. v_inactive_count: 364582. deficit: 0 2015 Oct 13 18:01:07: after-inactive-scan pass 2: page shortage: -37194, inactive page surplus: 9997, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:07: before-active-scan pass 2: page_shortage: -370441. 2015 Oct 13 18:01:07: after-active-scan pass 2: page_shortage: -370441. 2015 Oct 13 18:01:07: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253268. dtrace: cpu clocks out of sync (18130462652022 < 18130468761237; offset: 6109215). Results may be incorrect! 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253331. 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253331. 2015 Oct 13 18:01:08: before-inactive-scan: Scan goal 1: Move inactive to cache or free. 2015 Oct 13 18:01:08: before-inactive-scan: Page shortage: -34625, inactive page surplus: 10000 2015 Oct 13 18:01:08: v_free_target: 20888, v_free_count: 55513, v_cache_count: 0. 2015 Oct 13 18:01:08: v_inactive_target: 31332. v_inactive_count: 364663. deficit: 0 2015 Oct 13 18:01:08: after-inactive-scan pass 1: page shortage: -34645, inactive page surplus: 9980, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253286. 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253286. 2015 Oct 13 18:01:08: before-inactive-scan: Scan goal 2: Launder dirty pages. 2015 Oct 13 18:01:08: before-inactive-scan: Page shortage: -34395, inactive page surplus: 10000 2015 Oct 13 18:01:08: v_free_target: 20888, v_free_count: 55283, v_cache_count: 0. 2015 Oct 13 18:01:08: v_inactive_target: 31332. v_inactive_count: 364618. deficit: 0 2015 Oct 13 18:01:08: after-inactive-scan pass 2: page shortage: -34396, inactive page surplus: 9999, addl shortage: 0, vnodes skipped: 0. 2015 Oct 13 18:01:08: before-active-scan pass 2: page_shortage: -367391. 2015 Oct 13 18:01:08: after-active-scan pass 2: page_shortage: -367391. 2015 Oct 13 18:01:08: checked-inactive-pages: Inactive page surplus: 10000, above limit: 253285. 2015 Oct 13 18:01:09: checked-inactive-pages: Inactive page surplus: 10000, above limit: 256158. 2015 Oct 13 18:01:09: before-inactive-scan: Scan goal 3: Pageout dirty pages. 2015 Oct 13 18:01:09: before-inactive-scan: Page shortage: -34665, inactive page surplus: 10000 2015 Oct 13 18:01:09: v_free_target: 20888, v_free_count: 55553, v_cache_count: 0. 2015 Oct 13 18:01:09: v_inactive_target: 31332. v_inactive_count: 367490. deficit: 0 dtrace: cpu clocks out of sync (18131462608939 < 18131468746340; offset: 6137401). Results may be incorrect! Assertion failed: (timestamp >= dtp->dt_last_timestamp), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c, line 3352. Obtained from: ElectroBSD --- .../contrib/opensolaris/lib/libdtrace/common/dt_consume.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c index 80db09254dd0..af7724f102d2 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c @@ -3047,7 +3047,13 @@ dtrace_consume(dtrace_hdl_t *dtp, FILE *fp, break; timestamp = dt_buf_oldest(buf, dtp); - assert(timestamp >= dtp->dt_last_timestamp); + if (timestamp < dtp->dt_last_timestamp) { + warnx("cpu clocks out of sync " + "(%ju < %ju; offset: %ju). " + "Results may be incorrect!", + timestamp, dtp->dt_last_timestamp, + dtp->dt_last_timestamp - timestamp); + } dtp->dt_last_timestamp = timestamp; if (timestamp == buf->dtbd_timestamp) { -- 2.32.0 From e5ee88e68f09586fdde3fa4cab8847c1877e9696 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 16 Jun 2015 15:21:46 +0200 Subject: [PATCH 086/325] share/dtrace: Import geli-key-monitor Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/README | 3 +- share/dtrace/geli-key-monitor | 191 ++++++++++++++++++++++++++++++++++ 3 files changed, 194 insertions(+), 1 deletion(-) create mode 100755 share/dtrace/geli-key-monitor diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index c3ec2d9ce510..6ae53bf6cbff 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -8,6 +8,7 @@ SCRIPTS= blocking \ disklatency \ disklatencycmd \ + geli-key-monitor \ hotopen \ nfsattrstats \ nfsclienttime \ diff --git a/share/dtrace/README b/share/dtrace/README index 6855c6aee721..aeb2a06727b5 100644 --- a/share/dtrace/README +++ b/share/dtrace/README @@ -2,4 +2,5 @@ $FreeBSD$ This directory contains scripts for use with the DTrace system. These files and directories contain code generated by the FreeBSD -Project for use with DTrace on FreeBSD. +Project for use with DTrace on FreeBSD and code generated by the +ElectroBSD project for use with DTrace on ElectroBSD. diff --git a/share/dtrace/geli-key-monitor b/share/dtrace/geli-key-monitor new file mode 100755 index 000000000000..2cf4ba5c09be --- /dev/null +++ b/share/dtrace/geli-key-monitor @@ -0,0 +1,191 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * geli-key-monitor + * + * Traces GELI to print the beginning of various keys and warns + * about already known ones. For motivation and example output see: + * https://www.fabiankeil.de/gehacktes/geli-key-monitor/ + * + * Copyright (c) 2012 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +dtrace:::BEGIN +{ + algo[ 2] = "3DES-CBC"; + algo[ 3] = "Blowfish-CBC"; + algo[11] = "AES-CBC"; + algo[21] = "Camellia-CBC"; + algo[22] = "AES-XTS"; + + /* + * These are the first bytes of known weak keys generated with + * an unitialized sc->sc_ekey on a little-endian system. + */ + known_keys[0x06d800ef] = 1; + known_keys[0x252c6a6d] = 1; + known_keys[0xa72b9c7c] = 1; + known_keys[0xfde44683] = 1; + + G_ELI_FLAG_ENC_IVKEY = 0x00400000; + + g_eli_key_stats_available = 0; + g_eli_hold_stats_available = 0; + + /* + * Geli v5 and higher use a different key for each GB, + * so large disks have several thousand keys. + * Monitoring only the first few should do, keeps the + * output and reduces the likelihood of prevents dynvar + * drops. + */ + max_disk_keys_to_monitor = 4; + + printf("%Y: Monitoring geli keys (up to %d encryption keys per disk). %s\n", + walltimestamp, max_disk_keys_to_monitor, "Press CTRL-C to exit."); +} + +fbt::g_eli_mkey_propagate:entry +{ + self->sc = (struct g_eli_softc *)arg0; + self->mkey = (struct g_eli_key *)arg1; + + self->geom_name = stringof(self->sc->sc_geom->name); + self->monitored_keys = 0; + self->key_limit_reached = 0; + + this->algo = self->sc->sc_ealgo; + this->algo_name = (algo[this->algo] != NULL) ? algo[this->algo] : "Unregistered"; + this->geli_version = self->sc->sc_version; + + printf("%Y: %s: %s:%-6s: version: %d, algo: %s, flags: 0x%x (FLAG_ENC_IVKEY: %d) ", + walltimestamp, + self->geom_name, + probefunc, probename, + this->geli_version, this->algo_name, + self->sc->sc_flags, + (self->sc->sc_flags & G_ELI_FLAG_ENC_IVKEY) != 0 + ); + printf("mkey: %08.8x, sc_mkey: %08.8x, sc_ekey: %08.8x\n", + *(uint32_t *)(self->mkey), + *(uint32_t *)(self->sc->sc_mkey), + *(uint32_t *)(self->sc->sc_ekey)); +} + +fbt::g_eli_mkey_propagate:return +/self->sc != NULL/ +{ + printf("%Y: %s: %s:%-6s: mkey: %08.8x, sc_mkey: %08.8x, sc_ekey: %08.8x, sc_ekeys_allocated: %d\n", + walltimestamp, + self->geom_name, + probefunc, probename, + *(uint32_t *)(self->mkey), + *(uint32_t *)(self->sc->sc_mkey), + *(uint32_t *)(self->sc->sc_ekey), + self->sc->sc_ekeys_allocated); +} + +fbt::g_eli_key_fill:entry +{ + self->sc = (struct g_eli_softc *)arg0; + self->key = (struct g_eli_key *)arg1; + self->geom_name = stringof(self->sc->sc_geom->name); +} + +fbt::g_eli_key_fill: +/(self->sc != NULL) && !self->key_limit_reached/ +{ + this->key = *(uint32_t *)(self->key->gek_key); + printf("%Y: %s: %s:%-6s: key->gek_key: %8.8x (%d).%s\n", + walltimestamp, + self->geom_name, + probefunc, probename, + this->key, + self->monitored_keys, + known_keys[this->key] ? " Key looks familiar!" : ""); +} + +fbt::g_eli_key_fill:return +/(self->sc != NULL) && !self->key_limit_reached/ +{ + this->key = *(uint32_t *)(self->key->gek_key); + @g_eli_keys[self->monitored_keys, this->key, self->geom_name] = count(); + /* + * Register the generated key as known. + * Unless the provider is reattached we do not want to see it again. + */ + known_keys[this->key] = 1; + g_eli_key_stats_available = 1; + self->monitored_keys++; +} + +fbt::g_eli_key_fill:return +/(self->sc != NULL) && (self->sc->sc_ekeys_allocated == max_disk_keys_to_monitor - 1)/ +{ + printf("%Y: %s: %s:%-6s: Encryption key limit per disk reached.\n", + walltimestamp, self->geom_name, probefunc, probename); + self->key_limit_reached = 1; +} + +fbt::g_eli_key_hold:entry +{ + self->sc = (struct g_eli_softc *)arg0; + self->offset = (off_t)arg1; + self->blocksize = (size_t)arg2; + self->geom_name = stringof(self->sc->sc_geom->name); + + this->first_mkey_bytes = *(uint32_t *)(self->sc->sc_mkey); + this->first_ekey_bytes = *(uint32_t *)(self->sc->sc_ekey); + /* + * We only check the first bytes here, so false-positives are + * theoretically posible although unlikely. + */ + this->empty_ekey = (0 == this->first_ekey_bytes); + + this->geli_version = self->sc->sc_version; + this->algo = self->sc->sc_ealgo; + this->algo_name = (algo[this->algo] != NULL) ? algo[this->algo] : "Unregistered"; + + @g_eli_key_hold[self->geom_name, + this->geli_version, + this->algo_name, + this->algo, + this->first_mkey_bytes, + this->first_ekey_bytes] = count(); + + g_eli_hold_stats_available = 1; +} + +tick-60sec, +dtrace:::END +/g_eli_hold_stats_available/ +{ + printf("\n---------------------------------------------------------\n"); + printf("%Y: g_eli_key_hold() calls so far:\n", walltimestamp); + printf("%-20s %12s %18s %11s %11s %10s\n", + "Provider", "Geli version", "Algorithm", "mkey start", "ekey start", "calls"); + printa("%-20s %12d %13s (%2d) %08x %08x %@10d\n", @g_eli_key_hold); +} + +tick-60sec, +dtrace:::END +/g_eli_key_stats_available/ +{ + printf("\n%Y: g_eli_key count (only works for geli version 5 or higher):\n", walltimestamp); + printa("%@u #%04d %8.8x %-25s\n", @g_eli_keys); +} -- 2.32.0 From 73e490296c6838b4032e607f08a2569d85068952 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 5 Sep 2015 22:46:09 +0200 Subject: [PATCH 087/325] usr/share/dtrace: Add look-who-is-reaping It can be used to monitor and tune the ARC cache reaper. The stack trace obviously isn't particular useful anymore now that the reaper is called from a single location. Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/look-who-is-reaping | 58 ++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+) create mode 100755 share/dtrace/look-who-is-reaping diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 6ae53bf6cbff..62b91e24d1f9 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -10,6 +10,7 @@ SCRIPTS= blocking \ disklatencycmd \ geli-key-monitor \ hotopen \ + look-who-is-reaping \ nfsattrstats \ nfsclienttime \ siftr \ diff --git a/share/dtrace/look-who-is-reaping b/share/dtrace/look-who-is-reaping new file mode 100755 index 000000000000..0db6324a64f7 --- /dev/null +++ b/share/dtrace/look-who-is-reaping @@ -0,0 +1,58 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * look-who-is-reaping + * + * Collect stack traces for reap_arc_caches(). + * + * Copyright (c) 2015 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +dtrace:::BEGIN +{ + printf("%Y: Look who's (ARC cache) reaping .... Hit CTRL-C to exit.\n", walltimestamp); +} + +fbt::reap_arc_caches:entry +{ + @reapers[probefunc,stack(3)] = count(); + @reapers_total[probefunc,stack(3)] = count(); + reaped++ +} + +tick-10s +/reaped == 0/ +{ + printf("%Y: The ARC cache reaper has not been seen for 10 seconds!\n", walltimestamp); +} + +tick-10s +/reaped/ +{ + printf("%Y: The ARC reaper was called:\n", walltimestamp); + printa(@reapers); + trunc(@reapers); + reaped = 0 +} + +END +/reaped/ +{ + printf("%Y: The ARC reaper stats total:\n", walltimestamp); + printa(@reapers_total); +} -- 2.32.0 From 6af8e23e6b7760cd36660465ee32aaf2bd0d7306 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 4 Sep 2015 20:07:22 +0200 Subject: [PATCH 088/325] share/dtrace: Import fbt-time ... which llquantizes the time spent in a given kernel function Among other things this can be useful for general system tuning and analysis of lock contention (once you know where to expect it). Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/fbt-time | 80 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 81 insertions(+) create mode 100755 share/dtrace/fbt-time diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 62b91e24d1f9..1297a3317dc8 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -8,6 +8,7 @@ SCRIPTS= blocking \ disklatency \ disklatencycmd \ + fbt-time \ geli-key-monitor \ hotopen \ look-who-is-reaping \ diff --git a/share/dtrace/fbt-time b/share/dtrace/fbt-time new file mode 100755 index 000000000000..27a950b63a0c --- /dev/null +++ b/share/dtrace/fbt-time @@ -0,0 +1,80 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * fbt-time + * + * Measures the time spent in a given kernel function and llquantizes + * the results for the traced calls. Example output can be seen at: + * http://www.fabiankeil.de/gehacktes/electrobsd/zfs-arc-tuning/ + * + * WARNING: Tracing kernel functions that get executed several thousand + * times per second may negatively impact system performance + * and can even cause brief lockups. Before using this script + * on production systems you may want to sample the traced + * kernel function first. + * + * XXX: This script should be fleshed out to be more resilient to + * incorrect input and provide more awesome stats. A better name + * would be nice, too. + * + * Copyright (c) 2014 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +dtrace:::BEGIN +{ + traced_function = $$1; + + multiple = "microseconds"; + divisor = 1000; + executions = 0; + + this->timestamp = walltimestamp; + this->msecs = (this->timestamp / 1000000) % 1000; + printf("%Y.%.3d: Tracing '%s'. Hit CTRL-C to exit.\n", + this->timestamp, this->msecs, traced_function); +} + +fbt::$$1:entry +{ + self->timestamp = timestamp; + executions++; +} + +fbt::$$1:return +/self->timestamp/ +{ + this->elapsed = (timestamp - self->timestamp) / divisor; + @elapsed_time = llquantize(this->elapsed, 10, 0, 5, 10); + @elapsed_avg["Average"] = avg(this->elapsed); + self->elapsed = 0; +} + +tick-10sec, +END +/executions/ +{ + printf("%Y: Runtime for %s() in %s\n", walltimestamp, traced_function, multiple); + printa(@elapsed_avg); + printa(@elapsed_time); +} + +END +/executions == 0/ +{ + printf("%Y: Looks like %s() wasn't called yet.\n", walltimestamp, traced_function); +} -- 2.32.0 From 0b84437953783e7d5eac0e4d1fde15f86a7c273a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 15 May 2016 12:09:26 +0200 Subject: [PATCH 089/325] share/dtrace: Add geli-request-monitor ... which shows the BIO commands GELI received, their bio lengths from the caller's point of view and the maximum number of commands in flight. Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/geli-request-monitor | 91 +++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100755 share/dtrace/geli-request-monitor diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 1297a3317dc8..2705e9a87e9b 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -10,6 +10,7 @@ SCRIPTS= blocking \ disklatencycmd \ fbt-time \ geli-key-monitor \ + geli-request-monitor \ hotopen \ look-who-is-reaping \ nfsattrstats \ diff --git a/share/dtrace/geli-request-monitor b/share/dtrace/geli-request-monitor new file mode 100755 index 000000000000..70336060057d --- /dev/null +++ b/share/dtrace/geli-request-monitor @@ -0,0 +1,91 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * geli-request-monitor + * + * Shows the length distribution of various GELI commands from + * the callers point of view (that is, without geli overhead). + * + * Copyright (c) 2016-2017 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet + +dtrace:::BEGIN +{ + bio_command[0x01] = "BIO_READ"; + bio_command[0x02] = "BIO_WRITE"; + bio_command[0x03] = "BIO_DELETE"; + bio_command[0x04] = "BIO_GETATTR"; + bio_command[0x05] = "BIO_FLUSH"; + bio_command[0x06] = "BIO_CMD0"; + bio_command[0x07] = "BIO_CMD1"; + bio_command[0x08] = "BIO_CMD2"; + bio_command[0x09] = "BIO_ZONE"; +} + +fbt::g_eli_auth_run:entry, +fbt::g_eli_crypto_run:entry +{ + this->bp = (struct bio *)arg1; + this->sc = (struct g_eli_softc *)this->bp->bio_to->geom->softc; + + this->geom_name = stringof(this->sc->sc_geom->name); + this->length = this->bp->bio_length; + this->inflight = this->sc->sc_inflight; + + this->bio_command = (bio_command[this->bp->bio_cmd] != NULL) ? + bio_command[this->bp->bio_cmd] : "Unregistered command"; + + @inflight_max[this->geom_name] = max(this->inflight); + @inflight_max_total[this->geom_name] = max(this->inflight); + + @length[this->geom_name, this->bio_command] = + lquantize(this->length, 4096, 1052672, 4096); + @length_total[this->geom_name, this->bio_command] = + lquantize(this->length, 4096, 1052672, 4096); +/* + printf("%s: Length: %d, Inflight: %d", this->bio_command, + this->length, this->inflight); +*/ + + stats_available = 1; +} + +tick-60s +/stats_available/ +{ + printf("------------------------------\n"); + printf("Max requests inflight:\n"); + printa(@inflight_max); + printf("\nbp->bio_length:\n"); + printa(@length); + trunc(@length); + trunc(@inflight_max); + stats_available = 0; +} + +END +{ + /* Explicitly ditch interval stats so they aren't printed */ + trunc(@length); + trunc(@inflight_max); + + printf("\n------------------------------\n"); + printf("Max requests inflight:\n"); + printa(@inflight_max_total); + printf("\nbp->bio_length:\n"); + printa(@length_total); +} -- 2.32.0 From 6ece303bc603ef47971341a5820fdf8aeb0e9a73 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 20 Sep 2015 19:10:51 +0200 Subject: [PATCH 090/325] share/dtrace: Add monitor-page-scanner ... which, who would have guessed it, monitors the vm page scanner. It's useful to tune the sysctls for the inactive page limit. Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/monitor-page-scanner | 168 ++++++++++++++++++++++++++++++ 2 files changed, 169 insertions(+) create mode 100755 share/dtrace/monitor-page-scanner diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 2705e9a87e9b..123ea66735d3 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -13,6 +13,7 @@ SCRIPTS= blocking \ geli-request-monitor \ hotopen \ look-who-is-reaping \ + monitor-page-scanner \ nfsattrstats \ nfsclienttime \ siftr \ diff --git a/share/dtrace/monitor-page-scanner b/share/dtrace/monitor-page-scanner new file mode 100755 index 000000000000..85537caff8bf --- /dev/null +++ b/share/dtrace/monitor-page-scanner @@ -0,0 +1,168 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * monitor-page-scanner + * + * Traces the vm page scanner. + * + * Relies on SDT probes that currrently are not part of vanilla FreeBSD. + * + * Copyright (c) 2015 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +dtrace:::BEGIN +{ + goal[0] = "Update active LRU/deactivate pages"; + goal[1] = "Move inactive to cache or free"; + goal[2] = "Launder dirty pages"; + goal[3] = "Pageout dirty pages"; + start_time = walltimestamp; + + min_pass = 2; + + printf("%Y: Monitoring the page scanner. Minimum pass value to show 'boring' scans ", walltimestamp); + printf("without memory pressure or inactive page surplus: %d (%s). Press CTRL-C to abort.\n", + min_pass, goal[min_pass]); +} + +vm:kernel::checked-inactive-pages +{ + this->to_free = (int)arg0; +} + +vm:kernel::checked-inactive-pages +/this->to_free/ +{ + this->above_limit = (int)arg1; + + printf("%Y: %s: Inactive page surplus: %d, above limit: %d.\n", + walltimestamp, probename, this->to_free, this->above_limit); +} + +vm:kernel::before-inactive-scan +{ + this->pass = (int)arg1; + this->page_shortage = (int)arg2; + this->inactive_page_surplus = (int)arg4; + + /* + * Every pass code above 3 is treated like 3, + * adjust index accordingly. + */ + this->goal_index = (this->pass <= 3) ? this->pass : 3; + this->goal = goal[this->goal_index]; + @goals[this->goal] = count(); + @goals_total[this->goal] = count(); + last_pass[this->pass] = timestamp; +} + +vm:kernel::before-inactive-scan +/(this->pass >= min_pass) || (this->page_shortage > 0) +|| (this->inactive_page_surplus > 0)/ +{ + /* this->vmd = (struct vm_domain *)arg0; */ + this->deficit = (int)arg3; + + printf("%Y: %s: Scan goal %d: %s.\n", + walltimestamp, probename, this->pass, this->goal); + printf("%Y: %s: Page shortage: %d, inactive page surplus: %d\n", + walltimestamp, probename, this->page_shortage, this->inactive_page_surplus); + printf("%Y: v_free_target: %d, v_free_count: %d, v_cache_count: %d.\n", + walltimestamp, `vm_cnt.v_free_target, `vm_cnt.v_free_count, `vm_cnt.v_cache_count); + printf("%Y: v_inactive_target: %d. v_inactive_count: %d. deficit: %d\n", + walltimestamp, `vm_cnt.v_inactive_target, `vm_cnt.v_inactive_count, this->deficit); +} + +vm:kernel::after-inactive-scan +{ + this->pass = (int)arg1; + this->page_shortage = (int)arg2; + this->addl_page_shortage = (int)arg3; + this->vnodes_skipped = (int)arg4; + this->inactive_page_surplus = (int)arg5; +} + +vm:kernel::after-inactive-scan +/(this->pass >= min_pass) || (this->page_shortage > 0) || +(this->addl_page_shortage > 0) || (this->vnodes_skipped > 0) || +(this->inactive_page_surplus > 0)/ +{ + printf("%Y: %s pass %d: page shortage: %d, inactive page surplus: %d, addl shortage: %d, vnodes skipped: %d.\n", + walltimestamp, probename, this->pass, this->page_shortage, + this->inactive_page_surplus, this->addl_page_shortage, this->vnodes_skipped); +} + +vm:kernel::before-active-scan, +vm:kernel::after-active-scan +{ + this->pass = (int)arg1; + this->page_shortage = (int)arg2; +} + +vm:kernel::before-active-scan, +vm:kernel::after-active-scan +/(this->pass >= min_pass) || (this->page_shortage > 0)/ +{ + printf("%Y: %s pass %d: page_shortage: %d.\n", + walltimestamp, probename, this->pass, this->page_shortage); +} + +tick-60s +{ + printf("%Y: Scan goals in the previous minute:", walltimestamp); + printa(@goals); + trunc(@goals); +} + +tick-60s +/last_pass[1]/ +{ + this->pass = 1; + this->elapsed = (timestamp - last_pass[this->pass]) / 1000000000; + printf("%Y: Seconds since last '%s' pass: %d.\n", + walltimestamp, goal[this->pass], this->elapsed); +} + +tick-60s +/last_pass[2]/ +{ + this->pass = 2; + this->elapsed = (timestamp - last_pass[this->pass]) / 1000000000; + printf("%Y: Seconds since last '%s' pass: %d.\n", + walltimestamp, goal[this->pass], this->elapsed); +} + +tick-60s +/last_pass[3]/ +{ + this->pass = 3; + this->elapsed = (timestamp - last_pass[this->pass]) / 1000000000; + printf("%Y: Seconds since last '%s' pass: %d.\n", + walltimestamp, goal[this->pass], this->elapsed); +} + +END +{ + printf("%Y: Scan goals since start of script at %Y:", + walltimestamp, start_time); + printa(@goals_total); + + /* Clear aggregates so DTrace does not show them again. */ + trunc(@goals_total); + trunc(@goals); +} -- 2.32.0 From 9bf59af57520acac695886d0d90aad7eb262b6c3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 22 Apr 2017 13:41:41 +0200 Subject: [PATCH 091/325] share/dtrace: Add zfsdbg-msg ... which shows ZFS debug messages. Obtained from: ElectroBSD --- share/dtrace/Makefile | 3 ++- share/dtrace/zfsdbg-msg | 12 ++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100755 share/dtrace/zfsdbg-msg diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 123ea66735d3..4386cba69afe 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -21,7 +21,8 @@ SCRIPTS= blocking \ tcpdebug \ tcpstate \ tcptrack \ - udptrack + udptrack \ + zfsdbg-msg SCRIPTSDIR= ${SHAREDIR}/dtrace diff --git a/share/dtrace/zfsdbg-msg b/share/dtrace/zfsdbg-msg new file mode 100755 index 000000000000..80e65b2dbae5 --- /dev/null +++ b/share/dtrace/zfsdbg-msg @@ -0,0 +1,12 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * zfs-dbgmsg - Emit ZFS debug messages + ***************************************************************************/ + +#pragma D option quiet + +zfs-dbgmsg +{ + printf("%Y: %s\n", walltimestamp, stringof(arg0)) +} -- 2.32.0 From 6ec081ec04d4c6f3f14fd9161ba9604673f9bfd3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 30 Sep 2015 21:15:20 +0200 Subject: [PATCH 092/325] Add two DTrace scripts. WIP Obtained from: ElectroBSD --- share/dtrace/io-breakdown | 19 +++++++++++++++++++ share/dtrace/open-ios | 31 +++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+) create mode 100755 share/dtrace/io-breakdown create mode 100755 share/dtrace/open-ios diff --git a/share/dtrace/io-breakdown b/share/dtrace/io-breakdown new file mode 100755 index 000000000000..ee9e592f1bb5 --- /dev/null +++ b/share/dtrace/io-breakdown @@ -0,0 +1,19 @@ +#!/usr/sbin/dtrace -s + +/* This is the example from dtrace_io(4) */ + +#pragma D option quiet +#pragma D option dynvarsize=16m + +#pragma D option quiet + +io:::start +{ + @[args[1]->device_name, execname, pid] = sum(args[0]->bio_bcount); +} + +END +{ + printf("%10s %20s %10s %15s", "DEVICE", "APP", "PID", "BYTES"); + printa("%10s %20s %10d %15@d", @); +} diff --git a/share/dtrace/open-ios b/share/dtrace/open-ios new file mode 100755 index 000000000000..ee8bbe0d922d --- /dev/null +++ b/share/dtrace/open-ios @@ -0,0 +1,31 @@ +#!/usr/sbin/dtrace -s + +#pragma D option quiet +#pragma D option dynvarsize=16m + +dtrace:::BEGIN +{ + printf("Tracing... Hit Ctrl-C to end.\n"); +} + +io:::start +{ + printf("%Y: Device name\n", walltimestamp); + print(args[1]->device_name); + start_time[arg0] = timestamp; + @open_ios = count(); +} + +io:::done +/this->start = start_time[arg0]/ +{ + @done_ios = count(); +} + +tick-10s +{ + printf("%Y: Started ios", walltimestamp); + printa(@open_ios); + printf("%Y: Done ios", walltimestamp); + printa(@done_ios); +} -- 2.32.0 From 55115e13ea72c7edc48eb19372d30d0a8d35890b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 25 Jun 2015 16:46:35 +0200 Subject: [PATCH 093/325] release/Makefile: Note that the release process may silently fail Obtained from: ElectroBSD --- release/Makefile | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/release/Makefile b/release/Makefile index 7f46e08a42cd..7114d33e85bb 100644 --- a/release/Makefile +++ b/release/Makefile @@ -2,6 +2,14 @@ # # Makefile for building releases and release media. # +# Note that the release process is rather fragile and lots +# of variables depend on each other in mysterious and +# undocumented ways. +# +# If you don't get the magic right you should consider yourself +# lucky if the build fails, the usual outcome is a release that +# does not work. +# # User-driven targets: # cdrom: Builds release CD-ROM media (disc1.iso) # dvdrom: Builds release DVD-ROM media (dvd1.iso) -- 2.32.0 From 1c693c7d4737df72a2a4d812f5b4573c760ae328 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 25 Jun 2015 16:50:54 +0200 Subject: [PATCH 094/325] release/Makefile: Remove more stuff that does not apply to ElecroBSD from the disc1 target Obtained from: ElectroBSD --- release/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/release/Makefile b/release/Makefile index 7114d33e85bb..160708bef2cf 100644 --- a/release/Makefile +++ b/release/Makefile @@ -206,7 +206,6 @@ disc1: packagesystem do cp $${dist} ${.TARGET}/${DIST_TARBALL_DIR}; \ done # Set up installation environment - ln -fs /tmp/bsdinstall_etc/resolv.conf ${.TARGET}/etc/resolv.conf echo sendmail_enable=\"NONE\" > ${.TARGET}/etc/rc.conf echo hostid_enable=\"NO\" >> ${.TARGET}/etc/rc.conf echo keymap=\"de\" >> ${.TARGET}/etc/rc.conf -- 2.32.0 From 28ddcedcbba63be0c6e47bf6b62d68f906c971fe Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 25 Jun 2015 17:08:38 +0200 Subject: [PATCH 095/325] Makefile.inc1: Workaround passwd and friends missing in the base.txz Obtained from: ElectroBSD --- Makefile.inc1 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Makefile.inc1 b/Makefile.inc1 index 48c66e9e0e40..6e089ed4df7c 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1461,6 +1461,9 @@ distributeworld installworld stageworld: _installcheck_world .PHONY -e 's@(type=)@uid=0 gid=0 time=${EPOCH_DATE}.0 \1@' \ -e 's@//@/@g' -i '.bak' ${METALOG} .endif + @# Workaround for parts of etc mysteriously not being added below base. + @# As usual the location in dist is fine. + sed -e 's@^\./etc@./base/etc@' -i '.etc.bak' ${METALOG} .for dist in base ${EXTRA_DISTRIBUTIONS} @# For each file that exists in this dist, print the corresponding @# line from the METALOG. This relies on the fact that -- 2.32.0 From 9d79cda6f0b6b815978ace112aa7274bf14972c7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 3 Jul 2015 09:50:52 +0200 Subject: [PATCH 096/325] copyright.h: Use more insightful trademark information Obtained from: ElectroBSD --- sys/sys/copyright.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/sys/copyright.h b/sys/sys/copyright.h index 7aa81dc4bb2f..ef90ba2d44f7 100644 --- a/sys/sys/copyright.h +++ b/sys/sys/copyright.h @@ -40,7 +40,7 @@ /* Foundation */ #define TRADEMARK_Foundation \ - "FreeBSD is a registered trademark of The FreeBSD Foundation.\n" + "ElectroBSD ain't no registered trademark of The ElectroBSD Foundation (which does not exist).\n" /* Berkeley */ #define COPYRIGHT_UCB \ -- 2.32.0 From abbaaa3a707b67f3279549ab27b18cdc5f4a6f66 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 8 Jul 2015 17:43:31 +0200 Subject: [PATCH 097/325] sys/kern: Remove another FreeBSD reference from the boot messages Obtained from: ElectroBSD --- sys/kern/subr_smp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/subr_smp.c b/sys/kern/subr_smp.c index 5a26bd7bf9e7..ab33903f8328 100644 --- a/sys/kern/subr_smp.c +++ b/sys/kern/subr_smp.c @@ -168,7 +168,7 @@ mp_start(void *dummy) } cpu_mp_start(); - printf("FreeBSD/SMP: Multiprocessor System Detected: %d CPUs\n", + printf("SMP: Multiprocessor System Detected: %d CPUs\n", mp_ncpus); /* Provide a default for most architectures that don't have SMT/HTT. */ -- 2.32.0 From 24e585bf39eb8f45518c5945352d8c21d2fadb5a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 12 Jul 2015 13:11:28 +0200 Subject: [PATCH 098/325] tools/test/devrandom: Import arctest d9a5fc80, a wrapper around dieharder Obtained from: ElectroBSD --- tools/test/devrandom/arc4test | 278 ++++++++++++++++++++++++++++++++++ 1 file changed, 278 insertions(+) create mode 100755 tools/test/devrandom/arc4test diff --git a/tools/test/devrandom/arc4test b/tools/test/devrandom/arc4test new file mode 100755 index 000000000000..c824997a4a38 --- /dev/null +++ b/tools/test/devrandom/arc4test @@ -0,0 +1,278 @@ +#!/bin/sh + +############################################################################ +# arc4test +# +# Collects "entropy" and lets dieharder analyze it later on. The collected +# entropy is split into smaller files so the data collected in multiple +# runs can be easily interleaved and tested together. +# +# By default, entropy files are generated with arc4cat, a wrapper around +# arc4random_buf(3) which is suspected of "not returning very random data" +# between FreeBSD r273872 and r278907. +# +# So far it looks like the data may be "random enough" to pass the tests. +# +# Usage: +# arc4test build : Build arc4cat in $ARC4CAT_DIR +# arc4test collect : Collect potential entropy with arc4cat +# arc4test collect -d : Collect potential entropy with Dilbert PNRG +# arc4test analyze : Interleave collected entropy files and +# pipe them into dieharder. +# arc4test analyze -f : Try to spead up things by caching the interleaved +# entropy in a single file. Reuses the file if it +# already exists. +# arc4test remix : (Re)build an entropy cache file based on the +# previously collected entropy files. Roughfly +# doubles the required disk space but may significantly +# improve performance. +# arc4test cat : Dump interleaved entropy files to stdout +# +# Copyright (c) 2015 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +############################################################################ + +ARC4CAT_DIR=. +ARC4CAT="${ARC4CAT_DIR}/arc4cat" +# May not contain spaces etc. +ENTROPY_DIR="./entropy" +ENTROPY_SUBDIR_PREFIX="collection-" +# This is currently the block size for dd, setting it "too high" +# is not expected to work, however that's not good for input mixing +# later on anyway. +# +# Currently the entropy is split while it's being collected which is +# convenient from a programming point of view, but if the file size +# is small the interleave performance "may" (vulgo: will) suck. +ENTROPY_FILE_SIZE=4k +# Number of entropy files collected per run. If you increase ENTROPY_FILE_SIZE +# (or intend to do lots of collection runs) you may want to decrease this. +MAX_ENTROPY_FILES=100000 +ENTROPY_CACHE_FILE="${ENTROPY_DIR}/cached-entropy" + +prepare() { +} + +# The default entropy source +get_arc4cat_entropy() { + "${ARC4CAT}" +} + +# This is the reverse engineered PRNG from Dilbert strip 2001-10-25: +# http://dilbert.com/strip/2001-10-25 +# +# It is used instead of get_arc4cat_entropy() if the +# collect flag -d is set. +# +# According to the literature (see URL above) you can never be sure +# if the output is random, however the generator seems to fail all +# the dieharder tests and thus doesn't look nearly as good as Yarrow +# and Fortuna. +# +# Until this changes it will not be considered for ElectroBSD. +get_dilbert_entropy() { + while true; do + # The loop has been partially unrolled + # for increased performance! + echo -n "999999" + done +} + +get_shiny_new_entropy() { + local entropy_flag="${1}" + + if [ "${entropy_flag}" = "-d" ]; then + get_dilbert_entropy + else + get_arc4cat_entropy + fi +} + +collect_entropy() { + local entropy_flag \ + i entropy_file entropy_subdir + + entropy_flag="${1}" + i=0 + entropy_subdir="${ENTROPY_DIR}/${ENTROPY_SUBDIR_PREFIX}$(date +%s)" + + mkdir -p "${entropy_subdir}" + + ENTROPY_FILE_POSTFIX="" + + echo "Collecting ${MAX_ENTROPY_FILES} entropy files of size ${ENTROPY_FILE_SIZE} ..." + + # We don't call get_shiny_new_entropy() inside the loop as it + # would result in bits of entropy getting dropped on the floor + # between files. While we don't care about the "waste", we do + # care about not being able to test those bits later on. + get_shiny_new_entropy "${entropy_flag}" | while [ "${i}" -lt "${MAX_ENTROPY_FILES}" ]; do + entropy_file="$(printf "${entropy_subdir}/%.6i" "${i}")" + #echo "Creating ${entropy_file}" + dd bs="${ENTROPY_FILE_SIZE}" count=1 of="${entropy_file}" 2>/dev/null + i=$((i + 1)) + done +} + +create_entropy_cache() { + local \ + entropy_file + + entropy_file="${ENTROPY_DIR}/${LARGE_ENTROPY_FILE_NAME}" + + echo "Building a single entropy file '${ENTROPY_CACHE_FILE}' based on the files collected previously ..." 1>&2 + cat_collected_entropy > "${ENTROPY_CACHE_FILE}" +} + +replay_entropy() { + local fast_flag \ + entropy_file + + fast_flag="${1}" + + if [ "${fast_flag}" = "-f" ]; then + entropy_file="${ENTROPY_DIR}/${LARGE_ENTROPY_FILE_NAME}" + + if [ ! -f "${ENTROPY_CACHE_FILE}" ]; then + create_entropy_cache + fi + cat_entropy_cache + else + cat_collected_entropy + fi +} + +warn_about_entropy_reuse() { + echo "$0: Oh noes, we're out of collected entropy. Going back to the beginning." 1>&2 + echo "This shouldn't be a problem as long as no single test sees repeated data." 1>&2 +} + +cat_entropy_cache() { + while true; do + cat "${ENTROPY_CACHE_FILE}" + warn_about_entropy_reuse + done +} + +cat_collected_entropy() { + local \ + i f entropy_collections entropy_subdir entropy_file + + # XXX: Too fucking slow + #entropy_collections="$(find "${ENTROPY_DIR}/" -name "${ENTROPY_SUBDIR_PREFIX}*" -depth 1 -type 1)" + + # Not best practice but at least the performance doesn't suck + # and it works as expected. + + entropy_collections="${ENTROPY_DIR}/${ENTROPY_SUBDIR_PREFIX}"* + i=0 + while true; do + f="$(printf "%.6i" "${i}")" + for entropy_subdir in $entropy_collections; do + entropy_file="${entropy_subdir}/${f}" + #echo "Catting ${entropy_file}" + cat "${entropy_file}" || return 1 + done + i=$((i + 1)) + if [ "${i}" -eq "${MAX_ENTROPY_FILES}" ]; then + warn_about_entropy_reuse + i=0 + fi + done +} + +get_dieharder_tests() { + dieharder -l | awk '/-d/ {print $2}' +} + +analyze_collected_entropy() { + local fast_flag \ + test_number + + fast_flag="${1}" + + # We call replay_entropy() inside the loop to make sure + # the beginning of the collected entropy is checked by all tests + # (instead of having each test start at different offsets). + for test_number in $(get_dieharder_tests); do + replay_entropy ${fast_flag} | dieharder -g 200 -d "${test_number}" + done +} + +get_arc4cat_code() { + cat< +#include +#include + +int main(void) { + char buf[4096]; + + while (1) { + arc4random_buf(buf, sizeof(buf)); + write(1, buf, sizeof(buf)); + } +} +EOF +} + +build_arc4cat() { + mkdir -p "${ARC4CAT_DIR}" + cd "${ARC4CAT_DIR}" + get_arc4cat_code > arc4cat.c + make arc4cat + rm arc4cat.c +} + +usage() { + echo "Looks like you are doing it wrong. Try one of these:" + echo + echo "$0 analyze" + echo "$0 build" + echo "$0 cat" + echo "$0 collect" + return 1 +} + +main() { + local mode="${1}" + + shift + set -e + prepare + + case "${mode}" in + analyze) + analyze_collected_entropy "${@}" + ;; + build) + build_arc4cat + ;; + cat) + cat_collected_entropy + ;; + collect) + collect_entropy "${@}" + ;; + remix) + create_entropy_cache + ;; + *) + usage + ;; + esac +} + +main "${@}" -- 2.32.0 From 1c847a5c8865b73e0e17edbbf0d492d3e2d24a30 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 May 2015 19:06:37 +0200 Subject: [PATCH 099/325] release: Add ElectroBSD install instructions ... that aren't particularly verbose, though. Obtained from: ElectroBSD --- release/rc.local | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/release/rc.local b/release/rc.local index 8162cd1efff6..7acf5070ef5e 100755 --- a/release/rc.local +++ b/release/rc.local @@ -33,7 +33,7 @@ if [ $? -eq 0 ]; then else # Serial or other console echo - echo "Welcome to FreeBSD!" + echo "Welcome to ElectroBSD!" echo echo "Please choose the appropriate terminal type for your system." echo "Common console types are:" @@ -58,7 +58,19 @@ if [ -f /etc/installerconfig ]; then exit fi -dialog --backtitle "FreeBSD Installer" --title "Welcome" --extra-button --extra-label "Shell" --ok-label "Install" --cancel-label "Live CD" --yesno "Welcome to FreeBSD! Would you like to begin an installation or use the live CD?" 0 0 +local timeout=15 +if dialog --backtitle "ElectroBSD $(uname -m) on a stick in da house" \ + --title "Careful now, your data may be at risk." \ + --timeout ${timeout} \ + --yesno "To install ElectroBSD, execute cloudiatr after creating a cloudiatr.conf. Understood? You have ${timeout} seconds to respond." \ + 0 0; then + echo "Great. Good luck." +else + echo "That's very unfortunate. If you don't have backups you probably should not continue." +fi + +exit 0; + case $? in $DIALOG_OK) # Install -- 2.32.0 From 230a47b736ed72ad90d25a0b057472dacff64af3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 1 Aug 2015 11:42:55 +0200 Subject: [PATCH 100/325] sys/dev/vt: Default to showing splash CPU logo(s) on boot Obtained from: ElectroBSD --- sys/dev/vt/vt_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/vt/vt_core.c b/sys/dev/vt/vt_core.c index d4cbb038d31a..536b49131c81 100644 --- a/sys/dev/vt/vt_core.c +++ b/sys/dev/vt/vt_core.c @@ -141,7 +141,7 @@ static VT_SYSCTL_INT(kbd_panic, 0, "Enable request to panic. " /* Used internally, not a tunable. */ int vt_draw_logo_cpus; -VT_SYSCTL_INT(splash_cpu, 0, "Show logo CPUs during boot"); +VT_SYSCTL_INT(splash_cpu, 1, "Show logo CPUs during boot"); VT_SYSCTL_INT(splash_ncpu, 0, "Override number of logos displayed " "(0 = do not override)"); VT_SYSCTL_INT(splash_cpu_style, 2, "Draw logo style " -- 2.32.0 From 663f1b8d7faef7682714a6f11ce6b3e92b87beed Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 1 Aug 2015 16:59:55 +0200 Subject: [PATCH 101/325] sys/dev/vt: Default to showing Beastie instead of the orb/sextoy Obtained from: ElectroBSD --- sys/dev/vt/vt_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/vt/vt_core.c b/sys/dev/vt/vt_core.c index 536b49131c81..57a782d3d161 100644 --- a/sys/dev/vt/vt_core.c +++ b/sys/dev/vt/vt_core.c @@ -144,7 +144,7 @@ int vt_draw_logo_cpus; VT_SYSCTL_INT(splash_cpu, 1, "Show logo CPUs during boot"); VT_SYSCTL_INT(splash_ncpu, 0, "Override number of logos displayed " "(0 = do not override)"); -VT_SYSCTL_INT(splash_cpu_style, 2, "Draw logo style " +VT_SYSCTL_INT(splash_cpu_style, 1, "Draw logo style " "(0 = Alternate beastie, 1 = Beastie, 2 = Orb)"); VT_SYSCTL_INT(splash_cpu_duration, 10, "Hide logos after (seconds)"); -- 2.32.0 From 27c5dc90a6562dc52efa79aa36eceeaf9f5e6eb6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 2 Aug 2015 17:56:01 +0200 Subject: [PATCH 102/325] sys/modules: Detach a couple of ATM-related modules Obtained from: ElectroBSD --- sys/modules/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index f4b7d4f8a5bd..4942c706e93c 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -143,7 +143,7 @@ SUBDIR= \ geom \ ${_glxiic} \ ${_glxsb} \ - gpio \ + ${_gpio} \ hifn \ hme \ ${_hpt27xx} \ -- 2.32.0 From c3ed5951d81715ddabd920328a7bf60831105b23 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 2 Aug 2015 19:41:56 +0200 Subject: [PATCH 103/325] crypto/openssl: Update OPENSSL_VERSION_TEXT and break 'FIPS mode' Obtained from: ElectroBSD --- crypto/openssl/include/openssl/opensslv.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/crypto/openssl/include/openssl/opensslv.h b/crypto/openssl/include/openssl/opensslv.h index 6e9ad75755c8..cf3b97d1ae86 100644 --- a/crypto/openssl/include/openssl/opensslv.h +++ b/crypto/openssl/include/openssl/opensslv.h @@ -40,7 +40,12 @@ extern "C" { * major minor fix final patch/beta) */ # define OPENSSL_VERSION_NUMBER 0x101010cfL -# define OPENSSL_VERSION_TEXT "OpenSSL 1.1.1l-freebsd 24 Aug 2021" +# ifdef OPENSSL_FIPS +# error "Due to being silly, FIPS validation is not supported by ElectroBSD" +# else +# define OPENSSL_VERSION_TEXT "OpenSSL 1.1.1l-ElectroBSD 2021-08-24" +# endif +# define OPENSSL_VERSION_PTEXT " part of " OPENSSL_VERSION_TEXT /*- * The macros below are to be used for shared library (.so, .dll, ...) -- 2.32.0 From 4376161831a61df9e03994d4f10c55ab2da657ae Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 15 Aug 2015 11:29:39 +0200 Subject: [PATCH 104/325] ttys: Reduce default number of active terminals and mark the console as insecure Obtained from: ElectroBSD --- sbin/init/ttys.amd64 | 16 ++++++++-------- sbin/init/ttys.i386 | 16 ++++++++-------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/sbin/init/ttys.amd64 b/sbin/init/ttys.amd64 index b8392780ac9a..061b449ac743 100644 --- a/sbin/init/ttys.amd64 +++ b/sbin/init/ttys.amd64 @@ -27,17 +27,17 @@ # # If console is marked "insecure", then init will ask for the root password # when going to single-user mode. -console none unknown off secure +console none unknown off insecure # ttyv0 "/usr/libexec/getty Pc" xterm onifexists secure # Virtual terminals -ttyv1 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv2 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv3 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv4 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv5 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv6 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv7 "/usr/libexec/getty Pc" xterm onifexists secure +ttyv1 "/usr/libexec/getty Pc" xterm off secure +ttyv2 "/usr/libexec/getty Pc" xterm off secure +ttyv3 "/usr/libexec/getty Pc" xterm off secure +ttyv4 "/usr/libexec/getty Pc" xterm off secure +ttyv5 "/usr/libexec/getty Pc" xterm off secure +ttyv6 "/usr/libexec/getty Pc" xterm off secure +ttyv7 "/usr/libexec/getty Pc" xterm off secure ttyv8 "/usr/local/bin/xdm -nodaemon" xterm off secure # Serial terminals # The 'dialup' keyword identifies dialin lines to login, fingerd etc. diff --git a/sbin/init/ttys.i386 b/sbin/init/ttys.i386 index b8392780ac9a..061b449ac743 100644 --- a/sbin/init/ttys.i386 +++ b/sbin/init/ttys.i386 @@ -27,17 +27,17 @@ # # If console is marked "insecure", then init will ask for the root password # when going to single-user mode. -console none unknown off secure +console none unknown off insecure # ttyv0 "/usr/libexec/getty Pc" xterm onifexists secure # Virtual terminals -ttyv1 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv2 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv3 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv4 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv5 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv6 "/usr/libexec/getty Pc" xterm onifexists secure -ttyv7 "/usr/libexec/getty Pc" xterm onifexists secure +ttyv1 "/usr/libexec/getty Pc" xterm off secure +ttyv2 "/usr/libexec/getty Pc" xterm off secure +ttyv3 "/usr/libexec/getty Pc" xterm off secure +ttyv4 "/usr/libexec/getty Pc" xterm off secure +ttyv5 "/usr/libexec/getty Pc" xterm off secure +ttyv6 "/usr/libexec/getty Pc" xterm off secure +ttyv7 "/usr/libexec/getty Pc" xterm off secure ttyv8 "/usr/local/bin/xdm -nodaemon" xterm off secure # Serial terminals # The 'dialup' keyword identifies dialin lines to login, fingerd etc. -- 2.32.0 From 3d08ad3e752d8716a97184f646cffc97a6c4bcde Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 21:28:15 +0200 Subject: [PATCH 105/325] usr.sbin: Don't build uathload if MK_SOURCELESS_UCODE is defined It breaks the build if the source tree does not contain the firmware. Obtained from: ElectroBSD --- usr.sbin/Makefile | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index 4481b14cc008..57c66dccf04d 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -211,7 +211,12 @@ SUBDIR.${MK_TIMED}+= timed SUBDIR.${MK_TOOLCHAIN}+= config SUBDIR.${MK_TOOLCHAIN}+= crunch SUBDIR.${MK_UNBOUND}+= unbound +# MK_SOURCELESS_UCODE is supposed to affect kernel +# modules only but there is no matching define for +# userland stuff. +.if ${MK_SOURCELESS_UCODE} != "no" SUBDIR.${MK_USB}+= uathload +.endif SUBDIR.${MK_USB}+= uhsoctl SUBDIR.${MK_USB}+= usbconfig SUBDIR.${MK_USB}+= usbdump -- 2.32.0 From 868ef1ec5759e7d7fd33fba659556d93781ba68c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 21:33:19 +0200 Subject: [PATCH 106/325] Detach fwcontrol from the build It's pointless without firewire(4). Obtained from: ElectroBSD --- usr.sbin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index 57c66dccf04d..33e81b4f64ac 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -28,7 +28,6 @@ SUBDIR= adduser \ extattrctl \ fifolog \ fstyp \ - fwcontrol \ getfmac \ getpmac \ gstat \ -- 2.32.0 From 9cc0f5735c809bc9926957b81dc2058fcb373fb5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 23:58:46 +0200 Subject: [PATCH 107/325] sys/x86: Remove another FreeBSD reference from log messages Obtained from: ElectroBSD --- sys/x86/x86/mp_x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/x86/x86/mp_x86.c b/sys/x86/x86/mp_x86.c index 41dc36672135..941b37dcc37b 100644 --- a/sys/x86/x86/mp_x86.c +++ b/sys/x86/x86/mp_x86.c @@ -683,7 +683,7 @@ cpu_mp_announce(void) const char *hyperthread; struct topo_analysis topology; - printf("FreeBSD/SMP: "); + printf("ElectroBSD/SMP: "); if (topo_analyze(&topo_root, 1, &topology)) { printf("%d package(s)", topology.entities[TOPO_LEVEL_PKG]); if (topology.entities[TOPO_LEVEL_GROUP] > 1) @@ -704,7 +704,7 @@ cpu_mp_announce(void) printf("\n"); if (disabled_cpus) { - printf("FreeBSD/SMP Online: "); + printf("ElectroBSD/SMP Online: "); if (topo_analyze(&topo_root, 0, &topology)) { printf("%d package(s)", topology.entities[TOPO_LEVEL_PKG]); -- 2.32.0 From 68ddf30086a2f7262906367c5d4b5111c6bdb155 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 4 Jun 2016 11:38:43 +0200 Subject: [PATCH 108/325] Detach a couple of rc scripts that aren't relevant for ElectroBSD XXX: Among other things this detaches bgfsck which could still be considered relevant for the memstick images. Obtained from: ElectroBSD --- libexec/rc/rc.d/Makefile | 6 ------ 1 file changed, 6 deletions(-) diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index 699505f98770..6fc9ee63d8ad 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -14,7 +14,6 @@ CONFS= DAEMON \ addswap \ adjkerntz \ archdep \ - bgfsck \ ${_blacklistd} \ ${_bluetooth} \ bridge \ @@ -32,7 +31,6 @@ CONFS= DAEMON \ dhclient \ dmesg \ dumpon \ - fsck \ gbde \ geli \ geli2 \ @@ -66,7 +64,6 @@ CONFS= DAEMON \ mdconfig2 \ mountd \ msgs \ - natd \ netif \ netoptions \ netwait \ @@ -85,8 +82,6 @@ CONFS= DAEMON \ pflog \ pfsync \ powerd \ - ppp \ - pppoed \ pwcheck \ quota \ random \ @@ -103,7 +98,6 @@ CONFS= DAEMON \ savecore \ securelevel \ serial \ - sppp \ statd \ static_arp \ static_ndp \ -- 2.32.0 From a6554c05a86b27ae9b132f5147c184fa487a183d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 9 Jul 2015 14:55:50 +0200 Subject: [PATCH 109/325] etc/pkg: Add and install ElectroBSD.conf and a pubkey file ... for testing purposes. While at it, detach FreeBSD.conf and the corresponding fingerprint file from the build. Obtained from: ElectroBSD --- share/keys/pkg/trusted/Makefile | 2 +- .../keys/pkg/trusted/electrobsd-test-key.conf | 2 ++ usr.sbin/pkg/ElectroBSD.conf | 21 +++++++++++++++++++ usr.sbin/pkg/Makefile | 7 ++++++- usr.sbin/pkg/electrobsd-test-repo.pub | 14 +++++++++++++ 5 files changed, 44 insertions(+), 2 deletions(-) create mode 100644 share/keys/pkg/trusted/electrobsd-test-key.conf create mode 100644 usr.sbin/pkg/ElectroBSD.conf create mode 100644 usr.sbin/pkg/electrobsd-test-repo.pub diff --git a/share/keys/pkg/trusted/Makefile b/share/keys/pkg/trusted/Makefile index e55977c0a9a9..f55afc979969 100644 --- a/share/keys/pkg/trusted/Makefile +++ b/share/keys/pkg/trusted/Makefile @@ -1,6 +1,6 @@ # $FreeBSD$ -FILES= pkg.freebsd.org.2013102301 +FILES= electrobsd-test-key.conf FILESDIR= ${SHAREDIR}/keys/pkg/trusted FILESMODE= 644 diff --git a/share/keys/pkg/trusted/electrobsd-test-key.conf b/share/keys/pkg/trusted/electrobsd-test-key.conf new file mode 100644 index 000000000000..e81a919e0ab2 --- /dev/null +++ b/share/keys/pkg/trusted/electrobsd-test-key.conf @@ -0,0 +1,2 @@ +function: "sha256" +fingerprint: "8b1ef90f16ccc99342fb204f7ff57cdd31449e5ce1baa10c0b1b09593c44219d" diff --git a/usr.sbin/pkg/ElectroBSD.conf b/usr.sbin/pkg/ElectroBSD.conf new file mode 100644 index 000000000000..4bf82d992bee --- /dev/null +++ b/usr.sbin/pkg/ElectroBSD.conf @@ -0,0 +1,21 @@ +ElectroBSD: { + # Note that accessing this repository requires a local proxy + # that port forwards the incomming http requests to an authorized + # Tor client that forwards them to: http://gkpssb7hd77qznoa.onion/ + # + # To get the required secret you have to ask someone who knows it and + # come up with a good reason why you want to run unreproducible packages + # instead of building your packages from source. + # + # Example bootstrapping setup: + # Your server: pkg -> sshd + # | + # Your client: --> ssh -> privoxy -> tor + # | + # Outside your control: --> Tor network -> Tor HS + url: "http://127.0.0.1:8000/packages/${ABI}/", + mirror_type: "NONE", + signature_type: "pubkey", + pubkey: "/etc/pkg/electrobsd-test-repo.pub", + enabled: yes +} diff --git a/usr.sbin/pkg/Makefile b/usr.sbin/pkg/Makefile index 3e3b99372786..65105f979790 100644 --- a/usr.sbin/pkg/Makefile +++ b/usr.sbin/pkg/Makefile @@ -15,11 +15,16 @@ PKGCONFBRANCH?= latest . endif . endif .endif -PKGCONF?= FreeBSD.conf.${PKGCONFBRANCH} +PKGCONF?= ElectroBSD.conf CONFS= ${PKGCONF} CONFSNAME_${PKGCONF}= ${PKGCONF:C/\.conf.+$/.conf/} CONFSDIR= /etc/pkg CONFSMODE= 644 + +FILES= electrobsd-test-repo.pub +FILESDIR= /etc/pkg +FILESMODE= 644 + PROG= pkg SRCS= pkg.c dns_utils.c config.c MAN= pkg.7 diff --git a/usr.sbin/pkg/electrobsd-test-repo.pub b/usr.sbin/pkg/electrobsd-test-repo.pub new file mode 100644 index 000000000000..ee213cb062b1 --- /dev/null +++ b/usr.sbin/pkg/electrobsd-test-repo.pub @@ -0,0 +1,14 @@ +-----BEGIN PUBLIC KEY----- +MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAtwIgcSgQT2QJvxwDiKhh +xYidaxWaJL/EHXnYH8TpiujNeg8yZUCiT07MChw/wO3KrVY3SlhFkl8MHnwaFWNS +jSJFNQIHoVo8of0juDXhIMMY0DTVFVpSfgq+Ea0SrVZawQchcc7XODjp7mQOH0Nz +Z0FUpQ4z7TsdrPV/Kj/SQteK7Q0qAnDGT5GBxb+4m5qmNbv74GVpWmxJ5kC4OY3g +v8oTRZ5wG/mf5nKG+QXGKbcfoK+fd+NL9gp5awhvnp5cRo2+ZrQvnYKy5ym9O+jM +9hv5bL7OYIW7gErPdQd0+SFejcDg4TamY824krkjgIf7pnRVAuuSHiy8PaHmmNMd +StFo8IW6687C0X4gNuaPtR1TCkVZugrgev/w5GPp765HmE4WvrtPU4u9/EvWO2L6 +2bHmsJqNIGyq1lfHk/cx49CEhDPjRcYin8MKKzRFUR7CWiNDBz2GK6Lcfd78cG5I +HGJAthhi3rOOnLsTMuIQ/+GkVCDNwweLxSoLQUzGnheHqBjYyZSfZ8u9I/OKtwp3 +OMhGOD1/yuRaat+QER5VwiDP3AxcIlq6aNNytJW6l/kZZLVsr99YmROyYsTps0Zm +OXqxQFsZ6zzlYO95HP4ITJ7ubk84YPTvxbQ6LqsijqRI322aNvxxeEWyFxXpyCpp +uq50HtVoVX1a4ONsh/9l75sCAwEAAQ== +-----END PUBLIC KEY----- -- 2.32.0 From c96a33866db1e0623130887aa74f82a9edc4c7e0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 4 Mar 2015 14:22:43 +0100 Subject: [PATCH 110/325] Import rc.d/rether which randomizes MAC addresses ... provided ifconfig supports "either random". Obtained from: ElectroBSD --- libexec/rc/rc.d/Makefile | 1 + libexec/rc/rc.d/rether | 97 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+) create mode 100755 libexec/rc/rc.d/rether diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index 6fc9ee63d8ad..1f0ffed41e14 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -88,6 +88,7 @@ CONFS= DAEMON \ rarpd \ rctl \ resolv \ + rether \ root \ route6d \ routing \ diff --git a/libexec/rc/rc.d/rether b/libexec/rc/rc.d/rether new file mode 100755 index 000000000000..51e860353458 --- /dev/null +++ b/libexec/rc/rc.d/rether @@ -0,0 +1,97 @@ +#!/bin/sh +# +########################################################################### +# +# rether - Randomizes MAC addresses +# +# Add the following line to /etc/rc.conf to randomize the MAC +# address for all recognized network interfaces that got one +# at startup: +# +# rether_enable="YES" +# +# You can specify the interfaces manually like this: +# +# rether_interfaces="bge0 iwn0" +# +# By default rether runs before netif so cloned devices aren't +# expected to exist yet. In case of wlan clones the MAC address +# of the parent is inherited so as long as it's randomized the +# clone should be fine too. +# +# Note that MAC address mismatches between clone and parent may +# prevent the clone from working as expected. If you intend to +# run this script after the system is up, you may want to +# explicitly set rether_interfaces to skip clones. +# +# Rether requires an ifconfig version that understands "ether random". +# +########################################################################### +# +# Copyright (c) 2014 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ALL YOUR +# DATA IS BELONG TO THE SOFTWARE AND MAY BE EATEN BY IT. IF THAT IS NOT +# ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. +########################################################################### + +# PROVIDE: rether +# REQUIRE: FILESYSTEMS +# BEFORE: netif + +. /etc/rc.subr + +name="rether" +rcvar="rether_enable" + +rether_enable="${rether_enable-NO}" + +start_cmd="rether_start" +stop_cmd=":" + +# For the reason given above, it may make sense to skip cloned devices. +get_ethernet_interfaces() { + ifconfig -l ether +} + +get_ethernet_address() { + local interface="${1}" + + ifconfig "${interface}" ether | awk '/ether/ {print $2}' +} + +rether_start() { + local \ + interface \ + ethernet_address + + if [ -z "${rether_interfaces}" ]; then + rether_interfaces="$(get_ethernet_interfaces)" + fi + + echo "Randomizing MAC addresses for: ${rether_interfaces}" + + for interface in $rether_interfaces; + do + ethernet_address="$(get_ethernet_address "${interface}")" + + ifconfig "${interface}" ether random + + if [ "${ethernet_address}" = "$(get_ethernet_address "${interface}")" ]; then + echo "Failed to randomize MAC address for ${interface}: ${ethernet_address}" + fi + done +} + +load_rc_config "${name}" +run_rc_command "${1}" -- 2.32.0 From 81fe15e8e9123becea534a35416cff599ae101cf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 17 Sep 2015 17:20:31 +0200 Subject: [PATCH 111/325] share/mk/src.opts.mk: Detach LIB32 from the build Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 87304b8b5871..a59aefe66d3b 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -115,7 +115,6 @@ __DEFAULT_YES_OPTIONS = \ LDNS \ LDNS_UTILS \ LEGACY_CONSOLE \ - LIB32 \ LIBPTHREAD \ LIBTHR \ LLVM_COV \ @@ -237,6 +236,7 @@ __DEFAULT_NO_OPTIONS += \ GCOV \ IPFILTER \ ISCSI \ + LIB32 \ NDIS \ RBOOTD \ PC_SYSINSTALL \ -- 2.32.0 From f9dcad24e4db2fc19d883ae0bf85bcd53d2d7a05 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 22 Sep 2015 15:33:12 +0200 Subject: [PATCH 112/325] Attempt to fix MAC address randomization after r287197 Obtained from: ElectroBSD --- etc/Makefile | 1 + etc/mac-randomization.subr | 76 ++++++++++++++++++++++++++++++++++++++ libexec/rc/rc.d/netif | 3 ++ libexec/rc/rc.d/rether | 35 ++++-------------- 4 files changed, 87 insertions(+), 28 deletions(-) create mode 100755 etc/mac-randomization.subr diff --git a/etc/Makefile b/etc/Makefile index f33b026349ac..f485728b7967 100644 --- a/etc/Makefile +++ b/etc/Makefile @@ -15,6 +15,7 @@ SUBDIR+=sendmail BIN1= \ group \ login.access \ + mac-randomization.subr \ rc.bsdextended \ rc.firewall \ termcap.small diff --git a/etc/mac-randomization.subr b/etc/mac-randomization.subr new file mode 100755 index 000000000000..d486bf454591 --- /dev/null +++ b/etc/mac-randomization.subr @@ -0,0 +1,76 @@ +#!/bin/sh +# +########################################################################### +# +# Sub routines to randomizes MAC addresses after r287197 (WIP!) +# +# Add the following line to /etc/rc.conf to randomize the MAC +# address for all recognized network interfaces that got one +# at startup: +# +# rether_enable="YES" +# +# You can specify the interfaces manually like this: +# +# rether_interfaces="bge0 iwn0" +# +# Rether requires an ifconfig version that understands "ether random". +# +########################################################################### +# +# Copyright (c) 2014 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ALL YOUR +# DATA IS BELONG TO THE SOFTWARE AND MAY BE EATEN BY IT. IF THAT IS NOT +# ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. +########################################################################### + +rether_enable="${rether_enable-NO}" + +get_ethernet_interfaces() { + ifconfig -l ether +} + +get_ethernet_address() { + local interface="${1}" + + ifconfig "${interface}" ether | awk '/ether/ {print $2}' +} + +randomize_mac_addresses() { + local \ + interface \ + ethernet_address + + if [ "${rether_enable}" = "NO" ]; then + echo "Not randomizing any MAC addresses!" + return 0 + fi + + if [ -z "${rether_interfaces}" ]; then + rether_interfaces="$(get_ethernet_interfaces)" + fi + + echo "Randomizing MAC addresses for: ${rether_interfaces}" + + for interface in $rether_interfaces; + do + ethernet_address="$(get_ethernet_address "${interface}")" + + ifconfig "${interface}" ether random + + if [ "${ethernet_address}" = "$(get_ethernet_address "${interface}")" ]; then + echo "Failed to randomize MAC address for ${interface}: ${ethernet_address}" + fi + done +} diff --git a/libexec/rc/rc.d/netif b/libexec/rc/rc.d/netif index eb1efc4d6274..ff6962a1198e 100755 --- a/libexec/rc/rc.d/netif +++ b/libexec/rc/rc.d/netif @@ -32,6 +32,7 @@ . /etc/rc.subr . /etc/network.subr +. /etc/mac-randomization.subr name="netif" desc="Network interface setup" @@ -74,6 +75,8 @@ netif_start() # Create cloned interfaces clone_up $cmdifn + randomize_mac_addresses + # Rename interfaces. ifnet_rename $cmdifn diff --git a/libexec/rc/rc.d/rether b/libexec/rc/rc.d/rether index 51e860353458..d486bf454591 100755 --- a/libexec/rc/rc.d/rether +++ b/libexec/rc/rc.d/rether @@ -2,7 +2,7 @@ # ########################################################################### # -# rether - Randomizes MAC addresses +# Sub routines to randomizes MAC addresses after r287197 (WIP!) # # Add the following line to /etc/rc.conf to randomize the MAC # address for all recognized network interfaces that got one @@ -14,16 +14,6 @@ # # rether_interfaces="bge0 iwn0" # -# By default rether runs before netif so cloned devices aren't -# expected to exist yet. In case of wlan clones the MAC address -# of the parent is inherited so as long as it's randomized the -# clone should be fine too. -# -# Note that MAC address mismatches between clone and parent may -# prevent the clone from working as expected. If you intend to -# run this script after the system is up, you may want to -# explicitly set rether_interfaces to skip clones. -# # Rether requires an ifconfig version that understands "ether random". # ########################################################################### @@ -45,21 +35,8 @@ # ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. ########################################################################### -# PROVIDE: rether -# REQUIRE: FILESYSTEMS -# BEFORE: netif - -. /etc/rc.subr - -name="rether" -rcvar="rether_enable" - rether_enable="${rether_enable-NO}" -start_cmd="rether_start" -stop_cmd=":" - -# For the reason given above, it may make sense to skip cloned devices. get_ethernet_interfaces() { ifconfig -l ether } @@ -70,11 +47,16 @@ get_ethernet_address() { ifconfig "${interface}" ether | awk '/ether/ {print $2}' } -rether_start() { +randomize_mac_addresses() { local \ interface \ ethernet_address + if [ "${rether_enable}" = "NO" ]; then + echo "Not randomizing any MAC addresses!" + return 0 + fi + if [ -z "${rether_interfaces}" ]; then rether_interfaces="$(get_ethernet_interfaces)" fi @@ -92,6 +74,3 @@ rether_start() { fi done } - -load_rc_config "${name}" -run_rc_command "${1}" -- 2.32.0 From b9e7f8a4588cc76d770a825cd6e8932019848b22 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 29 Sep 2015 22:31:45 +0200 Subject: [PATCH 113/325] README: Spell out GPL correctly, the G does not stand for GNU Obtained from: ElectroBSD --- README | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README b/README index aad363baf9ea..7b6d20803c97 100644 --- a/README +++ b/README @@ -35,8 +35,8 @@ crypto Cryptography stuff (see crypto/README). etc Template files for /etc. -gnu Various commands and libraries under the GNU Public License. - Please see gnu/COPYING* for more information. +gnu Various commands and libraries under the GNU General Public + License. Please see gnu/COPYING* for more information. include System include files. -- 2.32.0 From ecb5ace0eb931a1af1848f3d9a7aed17a8d2aafc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 12:53:54 +0200 Subject: [PATCH 114/325] fortune/datafiles: Add German fundamental rights (poorly formatted) Enforcing these rights is left as an excercise for the reader. Obtained from: ElectroBSD --- usr.bin/fortune/datfiles/Makefile | 2 +- usr.bin/fortune/datfiles/grundrechte | 346 +++++++++++++++++++++++++++ 2 files changed, 347 insertions(+), 1 deletion(-) create mode 100644 usr.bin/fortune/datfiles/grundrechte diff --git a/usr.bin/fortune/datfiles/Makefile b/usr.bin/fortune/datfiles/Makefile index 80b8cd2ab641..ea87b1083a91 100644 --- a/usr.bin/fortune/datfiles/Makefile +++ b/usr.bin/fortune/datfiles/Makefile @@ -1,7 +1,7 @@ # @(#)Makefile 8.2 (Berkeley) 4/19/94 # $FreeBSD$ -DB= freebsd-tips +DB= grundrechte freebsd-tips BLDS= ${DB:S/$/.dat/} FILES= ${DB} ${BLDS} diff --git a/usr.bin/fortune/datfiles/grundrechte b/usr.bin/fortune/datfiles/grundrechte new file mode 100644 index 000000000000..c9a098878b17 --- /dev/null +++ b/usr.bin/fortune/datfiles/grundrechte @@ -0,0 +1,346 @@ +% +Grundgesetz Artikel 1 +(1) Die Würde des Menschen ist unantastbar. Sie zu achten und zu +schützen ist Verpflichtung aller staatlichen Gewalt. +(2) Das Deutsche Volk bekennt sich darum zu unverletzlichen und +unveräußerlichen Menschenrechten als Grundlage jeder menschlichen +Gemeinschaft, des Friedens und der Gerechtigkeit in der Welt. +(3) Die nachfolgenden Grundrechte binden Gesetzgebung, vollziehende +Gewalt und Rechtsprechung als unmittelbar geltendes Recht. + % +Grundgesetz Artikel 2 +(1) Jeder hat das Recht auf die freie Entfaltung seiner +Persönlichkeit, soweit er nicht die Rechte anderer verletzt und nicht +gegen die verfassungsmäßige Ordnung oder das Sittengesetz verstößt. +(2) Jeder hat das Recht auf Leben und körperliche Unversehrtheit. Die +Freiheit der Person ist unverletzlich. In diese Rechte darf nur auf +Grund eines Gesetzes eingegriffen werden. +% +Grundgesetz Artikel 3 +(1) Alle Menschen sind vor dem Gesetz gleich. +(2) Männer und Frauen sind gleichberechtigt. Der Staat fördert die +tatsächliche Durchsetzung der Gleichberechtigung von Frauen und +Männern und wirkt auf die Beseitigung bestehender Nachteile hin. +(3) Niemand darf wegen seines Geschlechtes, seiner Abstammung, seiner +Rasse, seiner Sprache, seiner Heimat und Herkunft, seines Glaubens, +seiner religiösen oder politischen Anschauungen benachteiligt oder +bevorzugt werden. Niemand darf wegen seiner Behinderung benachteiligt +werden. +% +Grundgesetz Artikel 4 +(1) Die Freiheit des Glaubens, des Gewissens und die Freiheit des +religiösen und weltanschaulichen Bekenntnisses sind unverletzlich. +(2) Die ungestörte Religionsausübung wird gewährleistet. +(3) Niemand darf gegen sein Gewissen zum Kriegsdienst mit der Waffe +gezwungen werden. Das Nähere regelt ein Bundesgesetz. +% +Grundgesetz Artikel 5 +(1) Jeder hat das Recht, seine Meinung in Wort, Schrift und Bild frei +zu äußern und zu verbreiten und sich aus allgemein zugänglichen +Quellen ungehindert zu unterrichten. Die Pressefreiheit und die +Freiheit der Berichterstattung durch Rundfunk und Film werden +gewährleistet. Eine Zensur findet nicht statt. +(2) Diese Rechte finden ihre Schranken in den Vorschriften der +allgemeinen Gesetze, den gesetzlichen Bestimmungen zum Schutze der +Jugend und in dem Recht der persönlichen Ehre. +(3) Kunst und Wissenschaft, Forschung und Lehre sind frei. Die +Freiheit der Lehre entbindet nicht von der Treue zur Verfassung. +% +Grundgesetz Artikel 6 +(1) Ehe und Familie stehen unter dem besonderen Schutze der staatlichen Ordnung. +(2) Pflege und Erziehung der Kinder sind das natürliche Recht der +Eltern und die zuvörderst ihnen obliegende Pflicht. Über ihre +Betätigung wacht die staatliche Gemeinschaft. +(3) Gegen den Willen der Erziehungsberechtigten dürfen Kinder nur auf +Grund eines Gesetzes von der Familie getrennt werden, wenn die +Erziehungsberechtigten versagen oder wenn die Kinder aus anderen +Gründen zu verwahrlosen drohen. +(4) Jede Mutter hat Anspruch auf den Schutz und die Fürsorge der Gemeinschaft. +(5) Den unehelichen Kindern sind durch die Gesetzgebung die gleichen +Bedingungen für ihre leibliche und seelische Entwicklung und ihre +Stellung in der Gesellschaft zu schaffen wie den ehelichen Kindern. +% +Grundgesetz Artikel 7 +(1) Das gesamte Schulwesen steht unter der Aufsicht des Staates. +(2) Die Erziehungsberechtigten haben das Recht, über die Teilnahme des +Kindes am Religionsunterricht zu bestimmen. +(3) Der Religionsunterricht ist in den öffentlichen Schulen mit +Ausnahme der bekenntnisfreien Schulen ordentliches +Lehrfach. Unbeschadet des staatlichen Aufsichtsrechtes wird der +Religionsunterricht in Übereinstimmung mit den Grundsätzen der +Religionsgemeinschaften erteilt. Kein Lehrer darf gegen seinen Willen +verpflichtet werden, Religionsunterricht zu erteilen. +(4) Das Recht zur Errichtung von privaten Schulen wird +gewährleistet. Private Schulen als Ersatz für öffentliche Schulen +bedürfen der Genehmigung des Staates und unterstehen den +Landesgesetzen. Die Genehmigung ist zu erteilen, wenn die privaten +Schulen in ihren Lehrzielen und Einrichtungen sowie in der +wissenschaftlichen Ausbildung ihrer Lehrkräfte nicht hinter den +öffentlichen Schulen zurückstehen und eine Sonderung der Schüler nach +den Besitzverhältnissen der Eltern nicht gefördert wird. Die +Genehmigung ist zu versagen, wenn die wirtschaftliche und rechtliche +Stellung der Lehrkräfte nicht genügend gesichert ist. +(5) Eine private Volksschule ist nur zuzulassen, wenn die +Unterrichtsverwaltung ein besonderes pädagogisches Interesse anerkennt +oder, auf Antrag von Erziehungsberechtigten, wenn sie als +Gemeinschaftsschule, als Bekenntnis- oder Weltanschauungsschule +errichtet werden soll und eine öffentliche Volksschule dieser Art in +der Gemeinde nicht besteht. +(6) Vorschulen bleiben aufgehoben. +% +Grundgesetz Artikel 8 +(1) Alle Deutschen haben das Recht, sich ohne Anmeldung oder Erlaubnis +friedlich und ohne Waffen zu versammeln. +(2) Für Versammlungen unter freiem Himmel kann dieses Recht durch +Gesetz oder auf Grund eines Gesetzes beschränkt werden. +% +Grundgesetz Artikel 9 +(1) Alle Deutschen haben das Recht, Vereine und Gesellschaften zu bilden. +(2) Vereinigungen, deren Zwecke oder deren Tätigkeit den Strafgesetzen +zuwiderlaufen oder die sich gegen die verfassungsmäßige Ordnung oder +gegen den Gedanken der Völkerverständigung richten, sind verboten. +(3) Das Recht, zur Wahrung und Förderung der Arbeits- und +Wirtschaftsbedingungen Vereinigungen zu bilden, ist für jedermann und +für alle Berufe gewährleistet. Abreden, die dieses Recht einschränken +oder zu behindern suchen, sind nichtig, hierauf gerichtete Maßnahmen +sind rechtswidrig. Maßnahmen nach den Artikeln 12a, 35 Abs. 2 und 3, +Artikel 87a Abs. 4 und Artikel 91 dürfen sich nicht gegen +Arbeitskämpfe richten, die zur Wahrung und Förderung der Arbeits- und +Wirtschaftsbedingungen von Vereinigungen im Sinne des Satzes 1 geführt +werden. +% +Grundgesetz Artikel 10 +(1) Das Briefgeheimnis sowie das Post- und Fernmeldegeheimnis sind +unverletzlich. +(2) Beschränkungen dürfen nur auf Grund eines Gesetzes angeordnet +werden. Dient die Beschränkung dem Schutze der freiheitlichen +demokratischen Grundordnung oder des Bestandes oder der Sicherung des +Bundes oder eines Landes, so kann das Gesetz bestimmen, daß sie dem +Betroffenen nicht mitgeteilt wird und daß an die Stelle des +Rechtsweges die Nachprüfung durch von der Volksvertretung bestellte +Organe und Hilfsorgane tritt. +% +Grundgesetz Artikel 11 +(1) Alle Deutschen genießen Freizügigkeit im ganzen Bundesgebiet. +(2) Dieses Recht darf nur durch Gesetz oder auf Grund eines Gesetzes +und nur für die Fälle eingeschränkt werden, in denen eine ausreichende +Lebensgrundlage nicht vorhanden ist und der Allgemeinheit daraus +besondere Lasten entstehen würden oder in denen es zur Abwehr einer +drohenden Gefahr für den Bestand oder die freiheitliche demokratische +Grundordnung des Bundes oder eines Landes, zur Bekämpfung von +Seuchengefahr, Naturkatastrophen oder besonders schweren +Unglücksfällen, zum Schutze der Jugend vor Verwahrlosung oder um +strafbaren Handlungen vorzubeugen, erforderlich ist. +% +Grundgesetz Artikel 12 +(1) Alle Deutschen haben das Recht, Beruf, Arbeitsplatz und +Ausbildungsstätte frei zu wählen. Die Berufsausübung kann durch Gesetz +oder auf Grund eines Gesetzes geregelt werden. +(2) Niemand darf zu einer bestimmten Arbeit gezwungen werden, außer im +Rahmen einer herkömmlichen allgemeinen, für alle gleichen öffentlichen +Dienstleistungspflicht. +(3) Zwangsarbeit ist nur bei einer gerichtlich angeordneten +Freiheitsentziehung zulässig. +% +Grundgesetz Artikel 12a +(1) Männer können vom vollendeten achtzehnten Lebensjahr an zum Dienst +in den Streitkräften, im Bundesgrenzschutz oder in einem +Zivilschutzverband verpflichtet werden. +(2) Wer aus Gewissensgründen den Kriegsdienst mit der Waffe +verweigert, kann zu einem Ersatzdienst verpflichtet werden. Die Dauer +des Ersatzdienstes darf die Dauer des Wehrdienstes nicht +übersteigen. Das Nähere regelt ein Gesetz, das die Freiheit der +Gewissensentscheidung nicht beeinträchtigen darf und auch eine +Möglichkeit des Ersatzdienstes vorsehen muß, die in keinem +Zusammenhang mit den Verbänden der Streitkräfte und des +Bundesgrenzschutzes steht. +(3) Wehrpflichtige, die nicht zu einem Dienst nach Absatz 1 oder 2 +herangezogen sind, können im Verteidigungsfalle durch Gesetz oder auf +Grund eines Gesetzes zu zivilen Dienstleistungen für Zwecke der +Verteidigung einschließlich des Schutzes der Zivilbevölkerung in +Arbeitsverhältnisse verpflichtet werden; Verpflichtungen in +öffentlich-rechtliche Dienstverhältnisse sind nur zur Wahrnehmung +polizeilicher Aufgaben oder solcher hoheitlichen Aufgaben der +öffentlichen Verwaltung, die nur in einem öffentlich-rechtlichen +Dienstverhältnis erfüllt werden können, zulässig. Arbeitsverhältnisse +nach Satz 1 können bei den Streitkräften, im Bereich ihrer Versorgung +sowie bei der öffentlichen Verwaltung begründet werden; +Verpflichtungen in Arbeitsverhältnisse im Bereiche der Versorgung der +Zivilbevölkerung sind nur zulässig, um ihren lebensnotwendigen Bedarf +zu decken oder ihren Schutz sicherzustellen. +(4) Kann im Verteidigungsfalle der Bedarf an zivilen Dienstleistungen +im zivilen Sanitäts- und Heilwesen sowie in der ortsfesten +militärischen Lazarettorganisation nicht auf freiwilliger Grundlage +gedeckt werden, so können Frauen vom vollendeten achtzehnten bis zum +vollendeten fünfundfünfzigsten Lebensjahr durch Gesetz oder auf Grund +eines Gesetzes zu derartigen Dienstleistungen herangezogen werden. Sie +dürfen auf keinen Fall zum Dienst mit der Waffe verpflichtet werden. +(5) Für die Zeit vor dem Verteidigungsfalle können Verpflichtungen +nach Absatz 3 nur nach Maßgabe des Artikels 80a Abs. 1 begründet +werden. Zur Vorbereitung auf Dienstleistungen nach Absatz 3, für die +besondere Kenntnisse oder Fertigkeiten erforderlich sind, kann durch +Gesetz oder auf Grund eines Gesetzes die Teilnahme an +Ausbildungsveranstaltungen zur Pflicht gemacht werden. Satz 1 findet +insoweit keine Anwendung. +(6) Kann im Verteidigungsfalle der Bedarf an Arbeitskräften für die in +Absatz 3 Satz 2 genannten Bereiche auf freiwilliger Grundlage nicht +gedeckt werden, so kann zur Sicherung dieses Bedarfs die Freiheit der +Deutschen, die Ausübung eines Berufs oder den Arbeitsplatz aufzugeben, +durch Gesetz oder auf Grund eines Gesetzes eingeschränkt werden. Vor +Eintritt des Verteidigungsfalles gilt Absatz 5 Satz 1 entsprechend. +% +Grundgesetz Artikel 13 +(1) Die Wohnung ist unverletzlich. +(2) Durchsuchungen dürfen nur durch den Richter, bei Gefahr im Verzuge +auch durch die in den Gesetzen vorgesehenen anderen Organe angeordnet +und nur in der dort vorgeschriebenen Form durchgeführt werden. +(3) Begründen bestimmte Tatsachen den Verdacht, daß jemand eine durch +Gesetz einzeln bestimmte besonders schwere Straftat begangen hat, so +dürfen zur Verfolgung der Tat auf Grund richterlicher Anordnung +technische Mittel zur akustischen Überwachung von Wohnungen, in denen +der Beschuldigte sich vermutlich aufhält, eingesetzt werden, wenn die +Erforschung des Sachverhalts auf andere Weise unverhältnismäßig +erschwert oder aussichtslos wäre. Die Maßnahme ist zu befristen. Die +Anordnung erfolgt durch einen mit drei Richtern besetzten +Spruchkörper. Bei Gefahr im Verzuge kann sie auch durch einen +einzelnen Richter getroffen werden. +(4) Zur Abwehr dringender Gefahren für die öffentliche Sicherheit, +insbesondere einer gemeinen Gefahr oder einer Lebensgefahr, dürfen +technische Mittel zur Überwachung von Wohnungen nur auf Grund +richterlicher Anordnung eingesetzt werden. Bei Gefahr im Verzuge kann +die Maßnahme auch durch eine andere gesetzlich bestimmte Stelle +angeordnet werden; eine richterliche Entscheidung ist unverzüglich +nachzuholen. +(5) Sind technische Mittel ausschließlich zum Schutze der bei einem +Einsatz in Wohnungen tätigen Personen vorgesehen, kann die Maßnahme +durch eine gesetzlich bestimmte Stelle angeordnet werden. Eine +anderweitige Verwertung der hierbei erlangten Erkenntnisse ist nur zum +Zwecke der Strafverfolgung oder der Gefahrenabwehr und nur zulässig, +wenn zuvor die Rechtmäßigkeit der Maßnahme richterlich festgestellt +ist; bei Gefahr im Verzuge ist die richterliche Entscheidung +unverzüglich nachzuholen. +(6) Die Bundesregierung unterrichtet den Bundestag jährlich über den +nach Absatz 3 sowie über den im Zuständigkeitsbereich des Bundes nach +Absatz 4 und, soweit richterlich überprüfungsbedürftig, nach Absatz 5 +erfolgten Einsatz technischer Mittel. Ein vom Bundestag gewähltes +Gremium übt auf der Grundlage dieses Berichts die parlamentarische +Kontrolle aus. Die Länder gewährleisten eine gleichwertige +parlamentarische Kontrolle. +(7) Eingriffe und Beschränkungen dürfen im übrigen nur zur Abwehr +einer gemeinen Gefahr oder einer Lebensgefahr für einzelne Personen, +auf Grund eines Gesetzes auch zur Verhütung dringender Gefahren für +die öffentliche Sicherheit und Ordnung, insbesondere zur Behebung der +Raumnot, zur Bekämpfung von Seuchengefahr oder zum Schutze gefährdeter +Jugendlicher vorgenommen werden. +% +Grundgesetz Artikel 14 +(1) Das Eigentum und das Erbrecht werden gewährleistet. Inhalt und +Schranken werden durch die Gesetze bestimmt. +(2) Eigentum verpflichtet. Sein Gebrauch soll zugleich dem Wohle der +Allgemeinheit dienen. +(3) Eine Enteignung ist nur zum Wohle der Allgemeinheit zulässig. Sie +darf nur durch Gesetz oder auf Grund eines Gesetzes erfolgen, das Art +und Ausmaß der Entschädigung regelt. Die Entschädigung ist unter +gerechter Abwägung der Interessen der Allgemeinheit und der +Beteiligten zu bestimmen. Wegen der Höhe der Entschädigung steht im +Streitfalle der Rechtsweg vor den ordentlichen Gerichten offen. +% +Grundgesetz Artikel 15 +Grund und Boden, Naturschätze und Produktionsmittel können zum Zwecke +der Vergesellschaftung durch ein Gesetz, das Art und Ausmaß der +Entschädigung regelt, in Gemeineigentum oder in andere Formen der +Gemeinwirtschaft überführt werden. Für die Entschädigung gilt Artikel +14 Abs. 3 Satz 3 und 4 entsprechend. +% +Grundgesetz Artikel 16 +(1) Die deutsche Staatsangehörigkeit darf nicht entzogen werden. Der +Verlust der Staatsangehörigkeit darf nur auf Grund eines Gesetzes und +gegen den Willen des Betroffenen nur dann eintreten, wenn der +Betroffene dadurch nicht staatenlos wird. +(2) Kein Deutscher darf an das Ausland ausgeliefert werden. Durch +Gesetz kann eine abweichende Regelung für Auslieferungen an einen +Mitgliedstaat der Europäischen Union oder an einen internationalen +Gerichtshof getroffen werden, soweit rechtsstaatliche Grundsätze +gewahrt sind. +% +Grundgesetz Artikel 16a +(1) Politisch Verfolgte genießen Asylrecht. +(2) Auf Absatz 1 kann sich nicht berufen, wer aus einem Mitgliedstaat +der Europäischen Gemeinschaften oder aus einem anderen Drittstaat +einreist, in dem die Anwendung des Abkommens über die Rechtsstellung +der Flüchtlinge und der Konvention zum Schutze der Menschenrechte und +Grundfreiheiten sichergestellt ist. Die Staaten außerhalb der +Europäischen Gemeinschaften, auf die die Voraussetzungen des Satzes 1 +zutreffen, werden durch Gesetz, das der Zustimmung des Bundesrates +bedarf, bestimmt. In den Fällen des Satzes 1 können +aufenthaltsbeendende Maßnahmen unabhängig von einem hiergegen +eingelegten Rechtsbehelf vollzogen werden. +(3) Durch Gesetz, das der Zustimmung des Bundesrates bedarf, können +Staaten bestimmt werden, bei denen auf Grund der Rechtslage, der +Rechtsanwendung und der allgemeinen politischen Verhältnisse +gewährleistet erscheint, daß dort weder politische Verfolgung noch +unmenschliche oder erniedrigende Bestrafung oder Behandlung +stattfindet. Es wird vermutet, daß ein Ausländer aus einem solchen +Staat nicht verfolgt wird, solange er nicht Tatsachen vorträgt, die +die Annahme begründen, daß er entgegen dieser Vermutung politisch +verfolgt wird. +(4) Die Vollziehung aufenthaltsbeendender Maßnahmen wird in den Fällen +des Absatzes 3 und in anderen Fällen, die offensichtlich unbegründet +sind oder als offensichtlich unbegründet gelten, durch das Gericht nur +ausgesetzt, wenn ernstliche Zweifel an der Rechtmäßigkeit der Maßnahme +bestehen; der Prüfungsumfang kann eingeschränkt werden und verspätetes +Vorbringen unberücksichtigt bleiben. Das Nähere ist durch Gesetz zu +bestimmen. +(5) Die Absätze 1 bis 4 stehen völkerrechtlichen Verträgen von +Mitgliedstaaten der Europäischen Gemeinschaften untereinander und mit +dritten Staaten nicht entgegen, die unter Beachtung der +Verpflichtungen aus dem Abkommen über die Rechtsstellung der +Flüchtlinge und der Konvention zum Schutze der Menschenrechte und +Grundfreiheiten, deren Anwendung in den Vertragsstaaten sichergestellt +sein muß, Zuständigkeitsregelungen für die Prüfung von Asylbegehren +einschließlich der gegenseitigen Anerkennung von Asylentscheidungen +treffen. +% +Grundgesetz Artikel 17 +Jedermann hat das Recht, sich einzeln oder in Gemeinschaft mit anderen +schriftlich mit Bitten oder Beschwerden an die zuständigen Stellen und +an die Volksvertretung zu wenden. +% +Grundgesetz Artikel 17a +(1) Gesetze über Wehrdienst und Ersatzdienst können bestimmen, daß für +die Angehörigen der Streitkräfte und des Ersatzdienstes während der +Zeit des Wehr- oder Ersatzdienstes das Grundrecht, seine Meinung in +Wort, Schrift und Bild frei zu äußern und zu verbreiten (Artikel 5 +Abs. 1 Satz 1 erster Halbsatz), das Grundrecht der +Versammlungsfreiheit (Artikel 8) und das Petitionsrecht (Artikel 17), +soweit es das Recht gewährt, Bitten oder Beschwerden in Gemeinschaft +mit anderen vorzubringen, eingeschränkt werden. +(2) Gesetze, die der Verteidigung einschließlich des Schutzes der +Zivilbevölkerung dienen, können bestimmen, daß die Grundrechte der +Freizügigkeit (Artikel 11) und der Unverletzlichkeit der Wohnung +(Artikel 13) eingeschränkt werden. +% +Grundgesetz Artikel 18 +Wer die Freiheit der Meinungsäußerung, insbesondere die Pressefreiheit +(Artikel 5 Abs. 1), die Lehrfreiheit (Artikel 5 Abs. 3), die +Versammlungsfreiheit (Artikel 8), die Vereinigungsfreiheit (Artikel +9), das Brief-, Post- und Fernmeldegeheimnis (Artikel 10), das +Eigentum (Artikel 14) oder das Asylrecht (Artikel 16a) zum Kampfe +gegen die freiheitliche demokratische Grundordnung mißbraucht, +verwirkt diese Grundrechte. Die Verwirkung und ihr Ausmaß werden durch +das Bundesverfassungsgericht ausgesprochen. +% +Grundgesetz Artikel 19 +(1) Soweit nach diesem Grundgesetz ein Grundrecht durch Gesetz oder +auf Grund eines Gesetzes eingeschränkt werden kann, muß das Gesetz +allgemein und nicht nur für den Einzelfall gelten. Außerdem muß das +Gesetz das Grundrecht unter Angabe des Artikels nennen. +(2) In keinem Falle darf ein Grundrecht in seinem Wesensgehalt +angetastet werden. +(3) Die Grundrechte gelten auch für inländische juristische Personen, +soweit sie ihrem Wesen nach auf diese anwendbar sind. +(4) Wird jemand durch die öffentliche Gewalt in seinen Rechten +verletzt, so steht ihm der Rechtsweg offen. Soweit eine andere +Zuständigkeit nicht begründet ist, ist der ordentliche Rechtsweg +gegeben. Artikel 10 Abs. 2 Satz 2 bleibt unberührt. +% -- 2.32.0 From 505c7b812ad4b372f2e9a47bcb0c418a4fe4f39a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 17 Aug 2015 13:16:50 +0200 Subject: [PATCH 115/325] share/skel/dot.profile: Change fortune file to "grundrechte" Obtained from: ElectroBSD --- share/skel/dot.profile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/skel/dot.profile b/share/skel/dot.profile index 02623293b592..dc3c0e9b8e60 100644 --- a/share/skel/dot.profile +++ b/share/skel/dot.profile @@ -27,4 +27,4 @@ if [ "$PWD" != "$HOME" ] && [ "$PWD" -ef "$HOME" ] ; then cd ; fi if [ -x /usr/bin/resizewin ] ; then /usr/bin/resizewin -z ; fi # Display a random cookie on each login. -if [ -x /usr/bin/fortune ] ; then /usr/bin/fortune freebsd-tips ; fi +if [ -x /usr/bin/fortune ] ; then /usr/bin/fortune grundrechte ; fi -- 2.32.0 From b7f5f375b7d1187cc644efc91d74ab06e7550d2b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 18 Aug 2015 09:57:30 +0200 Subject: [PATCH 116/325] sys/x86: Note that ElectroBSD does not protect against compromised hosts ... if a hypervisor has been detected. Obtained from: ElectroBSD --- sys/x86/x86/identcpu.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/sys/x86/x86/identcpu.c b/sys/x86/x86/identcpu.c index 2907575377b8..ce9bf2ae5d3b 100644 --- a/sys/x86/x86/identcpu.c +++ b/sys/x86/x86/identcpu.c @@ -2551,6 +2551,11 @@ static void print_hypervisor_info(void) { - if (*hv_vendor) - printf("Hypervisor: Origin = \"%s\"\n", hv_vendor); + if (*hv_vendor == '\0') + return; + + printf("Hypervisor: Origin = \"%s\"\n", hv_vendor); + printf("ElectroBSD security advise: It looks like you may be clown computing.\n" + "Please note that the people controlling the 'cloud' this system is\n" + "running on can monitor and sabotage everything it is doing.\n"); } -- 2.32.0 From a047ebc2728f95737b2fa556f7433b56bebb6483 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 8 Jan 2017 23:11:47 +0100 Subject: [PATCH 117/325] Detach enigma(1) from the build Obtained from: ElectroBSD --- usr.bin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/usr.bin/Makefile b/usr.bin/Makefile index 89ca5d01e433..cf506950632c 100644 --- a/usr.bin/Makefile +++ b/usr.bin/Makefile @@ -35,7 +35,6 @@ SUBDIR= alias \ elf2aout \ elfctl \ elfdump \ - enigma \ env \ etdump \ expand \ -- 2.32.0 From 4a14fb1368c28bf9fc82feb1b8f3080ddcaafda9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 9 Jan 2016 00:37:52 +0100 Subject: [PATCH 118/325] crontab: Default to not sending out (most-likely unencrypted) mails Obtained from: ElectroBSD --- usr.sbin/cron/cron/crontab | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/usr.sbin/cron/cron/crontab b/usr.sbin/cron/cron/crontab index 994d4ae6af00..cbcff76d64bf 100644 --- a/usr.sbin/cron/cron/crontab +++ b/usr.sbin/cron/cron/crontab @@ -1,9 +1,8 @@ -# /etc/crontab - root's crontab for FreeBSD -# -# $FreeBSD$ +# /etc/crontab - root's crontab for ElectroBSD # SHELL=/bin/sh PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin +MAILTO="" # #minute hour mday month wday who command # -- 2.32.0 From adfdaaff1c79ae3ef2c0f6e923f9a97e0b99d794 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 16 Jan 2016 15:24:51 +0100 Subject: [PATCH 119/325] Detach unreproducible uzip tests (added in r293821) that aren't relevant for ElectroBSD Obtained from: ElectroBSD --- tests/sys/geom/class/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/sys/geom/class/Makefile b/tests/sys/geom/class/Makefile index 6e573d6164b6..beae0fd3d675 100644 --- a/tests/sys/geom/class/Makefile +++ b/tests/sys/geom/class/Makefile @@ -18,7 +18,8 @@ TESTS_SUBDIRS+= part TESTS_SUBDIRS+= raid3 TESTS_SUBDIRS+= shsec TESTS_SUBDIRS+= stripe -TESTS_SUBDIRS+= uzip +# Currently not reproducible +#TESTS_SUBDIRS+= uzip ${PACKAGE}FILES+= geom_subr.sh -- 2.32.0 From 82ddb792f67856054e4b857cde1d803c59f3f7e6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 21 Jan 2016 12:49:29 +0100 Subject: [PATCH 120/325] OpenSSH: Disable DSA keys by default ... as it's done by vanilla OpenSSH since 6.9p1. This reverts FreeBSD r294495 which enabled them again due to POLA concerns that aren't relevant for ElectroBSD. Obtained from: ElectroBSD --- UPDATING | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/UPDATING b/UPDATING index 88a610e4705c..01dd8fc37776 100644 --- a/UPDATING +++ b/UPDATING @@ -1158,6 +1158,10 @@ from older version of current across the gcc/clang cutover is a bit fragile. restored by setting vfs.zfs.vol.recursive=1. 20160119: + The default configuration of ssh(1) no longer allows to use ssh-dss + keys. To enable using them, add 'ssh-dss' to PubkeyAcceptedKeyTypes + option in the /etc/ssh/ssh_config. Refer to ssh_config(5) for more + information. The NONE and HPN patches has been removed from OpenSSH. They are still available in the security/openssh-portable port. -- 2.32.0 From 255b7c46ae4dc45681510146545707a410d8cbfb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Mar 2016 14:27:33 +0100 Subject: [PATCH 121/325] OpenSSH: Remove AES-CBC ciphers from the default cipher list on the server As it's done in OpenSSH 6.7p1 and later. This reverts FreeBSD r296634. Obtained from: ElectroBSD --- crypto/openssh/FREEBSD-upgrade | 8 +------- crypto/openssh/myproposal.h | 3 +-- crypto/openssh/sshd_config.5 | 3 +-- 3 files changed, 3 insertions(+), 11 deletions(-) diff --git a/crypto/openssh/FREEBSD-upgrade b/crypto/openssh/FREEBSD-upgrade index 244042ec83e4..f1921a5d533e 100644 --- a/crypto/openssh/FREEBSD-upgrade +++ b/crypto/openssh/FREEBSD-upgrade @@ -1,3 +1,4 @@ + FreeBSD maintainer's guide to OpenSSH-portable ============================================== @@ -157,13 +158,6 @@ ignore HPN-related configuration options to avoid breaking existing configurations. -9) AES-CBC - - The AES-CBC ciphers were removed from the server-side proposal list - in 6.7p1 due to theoretical weaknesses and the availability of - superior ciphers (including AES-CTR and AES-GCM). We have re-added - them for compatibility with third-party clients. - This port was brought to you by (in no particular order) DARPA, NAI diff --git a/crypto/openssh/myproposal.h b/crypto/openssh/myproposal.h index d5a1b7a2fa24..fad841cc736f 100644 --- a/crypto/openssh/myproposal.h +++ b/crypto/openssh/myproposal.h @@ -122,8 +122,7 @@ #define KEX_SERVER_ENCRYPT \ "chacha20-poly1305@openssh.com," \ "aes128-ctr,aes192-ctr,aes256-ctr" \ - AESGCM_CIPHER_MODES \ - ",aes128-cbc,aes192-cbc,aes256-cbc" + AESGCM_CIPHER_MODES #define KEX_CLIENT_ENCRYPT KEX_SERVER_ENCRYPT diff --git a/crypto/openssh/sshd_config.5 b/crypto/openssh/sshd_config.5 index 366669cec5ad..0ae5e665af3d 100644 --- a/crypto/openssh/sshd_config.5 +++ b/crypto/openssh/sshd_config.5 @@ -495,8 +495,7 @@ The default is: .Bd -literal -offset indent chacha20-poly1305@openssh.com, aes128-ctr,aes192-ctr,aes256-ctr, -aes128-gcm@openssh.com,aes256-gcm@openssh.com, -aes128-cbc,aes192-cbc,aes256-cbc +aes128-gcm@openssh.com,aes256-gcm@openssh.com .Ed .Pp The list of available ciphers may also be obtained using -- 2.32.0 From 3d14f55ff6cb961326dd75124c73af2d6b2c187e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Mar 2016 14:53:42 +0100 Subject: [PATCH 122/325] sshd: Disable UsePam, X11Forwarding and UseDNS by default Obtained from: ElectroBSD --- crypto/openssh/servconf.c | 6 +++--- crypto/openssh/sshd_config | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/crypto/openssh/servconf.c b/crypto/openssh/servconf.c index 8b07e4b923ae..9588ae0a83a6 100644 --- a/crypto/openssh/servconf.c +++ b/crypto/openssh/servconf.c @@ -269,7 +269,7 @@ fill_default_server_options(ServerOptions *options) /* Portable-specific options */ if (options->use_pam == -1) - options->use_pam = 1; + options->use_pam = 0; /* Standard Options */ if (options->num_host_key_files == 0) { @@ -313,7 +313,7 @@ fill_default_server_options(ServerOptions *options) if (options->print_lastlog == -1) options->print_lastlog = 1; if (options->x11_forwarding == -1) - options->x11_forwarding = 1; + options->x11_forwarding = 0; if (options->x11_display_offset == -1) options->x11_display_offset = 10; if (options->x11_use_localhost == -1) @@ -389,7 +389,7 @@ fill_default_server_options(ServerOptions *options) if (options->max_sessions == -1) options->max_sessions = DEFAULT_SESSIONS_MAX; if (options->use_dns == -1) - options->use_dns = 1; + options->use_dns = 0; if (options->client_alive_interval == -1) options->client_alive_interval = 0; if (options->client_alive_count_max == -1) diff --git a/crypto/openssh/sshd_config b/crypto/openssh/sshd_config index 4f636bc5c13b..10361b466257 100644 --- a/crypto/openssh/sshd_config +++ b/crypto/openssh/sshd_config @@ -83,12 +83,12 @@ AuthorizedKeysFile .ssh/authorized_keys # If you just want the PAM account and session checks to run without # PAM authentication, then enable this but set PasswordAuthentication # and ChallengeResponseAuthentication to 'no'. -#UsePAM yes +#UsePAM no #AllowAgentForwarding yes #AllowTcpForwarding yes #GatewayPorts no -#X11Forwarding yes +#X11Forwarding no #X11DisplayOffset 10 #X11UseLocalhost yes #PermitTTY yes @@ -99,7 +99,7 @@ AuthorizedKeysFile .ssh/authorized_keys #Compression delayed #ClientAliveInterval 0 #ClientAliveCountMax 3 -#UseDNS yes +#UseDNS no #PidFile /var/run/sshd.pid #MaxStartups 10:30:100 #PermitTunnel no -- 2.32.0 From cdf2bd2bd86030ce85a3a3adb2631c8dc58e9ce6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Mar 2016 14:42:00 +0100 Subject: [PATCH 123/325] OpenSSH ssh and sshd: Default to not sending an VersionAddendum This is also the default used by OpenSSH upstream. Obtained from: ElectroBSD --- crypto/openssh/ssh_config | 2 +- crypto/openssh/sshd_config | 3 ++- crypto/openssh/version.h | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/crypto/openssh/ssh_config b/crypto/openssh/ssh_config index e7d969abed25..a7a1570ebe9a 100644 --- a/crypto/openssh/ssh_config +++ b/crypto/openssh/ssh_config @@ -46,4 +46,4 @@ # ProxyCommand ssh -q -W %h:%p gateway.example.com # RekeyLimit 1G 1h # VerifyHostKeyDNS yes -# VersionAddendum FreeBSD-20200214 +# VersionAddendum none diff --git a/crypto/openssh/sshd_config b/crypto/openssh/sshd_config index 10361b466257..ea73208f3079 100644 --- a/crypto/openssh/sshd_config +++ b/crypto/openssh/sshd_config @@ -105,7 +105,8 @@ AuthorizedKeysFile .ssh/authorized_keys #PermitTunnel no #ChrootDirectory none #UseBlacklist no -#VersionAddendum FreeBSD-20200214 +#VersionAddendum none + # no default banner path #Banner none diff --git a/crypto/openssh/version.h b/crypto/openssh/version.h index 9dbaf16e7192..92c44c68d07d 100644 --- a/crypto/openssh/version.h +++ b/crypto/openssh/version.h @@ -6,7 +6,7 @@ #define SSH_PORTABLE "p1" #define SSH_RELEASE SSH_VERSION SSH_PORTABLE -#define SSH_VERSION_FREEBSD "FreeBSD-20200214" +#define SSH_VERSION_FREEBSD "" #ifdef WITH_OPENSSL #define OPENSSL_VERSION_STRING OpenSSL_version(OPENSSL_VERSION) -- 2.32.0 From 1fbc1e891d8f0d4a7c0ee803ba086fe9763fd6a4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 Feb 2016 13:15:19 +0100 Subject: [PATCH 124/325] include/Makefile: Unbreak build without atm sources Obtained from: ElectroBSD --- include/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/Makefile b/include/Makefile index bd0ddc10b492..f341f77825d2 100644 --- a/include/Makefile +++ b/include/Makefile @@ -54,7 +54,7 @@ LSUBDIRS= cam/ata cam/mmc cam/nvme cam/scsi \ geom/mirror geom/mountver geom/multipath geom/nop \ geom/raid geom/raid3 geom/shsec geom/stripe geom/virstor \ net/altq \ - netgraph/atm netgraph/netflow \ + netgraph/netflow \ netinet/cc \ netinet/netdump \ security/audit \ -- 2.32.0 From 31d8e7beda5cc1cdee17ac2c982955949872b9fb Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 24 Mar 2016 16:54:42 +0100 Subject: [PATCH 125/325] sys/netinet: Default to not processing IP options Obtained from: ElectroBSD --- sys/netinet/ip_options.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/netinet/ip_options.c b/sys/netinet/ip_options.c index 393cfe0b1178..b4e9ea4997f6 100644 --- a/sys/netinet/ip_options.c +++ b/sys/netinet/ip_options.c @@ -80,7 +80,7 @@ SYSCTL_INT(_net_inet_ip, IPCTL_ACCEPTSOURCEROUTE, accept_sourceroute, "Enable accepting source routed IP packets"); #define V_ip_acceptsourceroute VNET(ip_acceptsourceroute) -VNET_DEFINE(int, ip_doopts) = 1; /* 0 = ignore, 1 = process, 2 = reject */ +VNET_DEFINE(int, ip_doopts) = 0; /* 0 = ignore, 1 = process, 2 = reject */ SYSCTL_INT(_net_inet_ip, OID_AUTO, process_options, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_doopts), 0, "Enable IP options processing ([LS]SRR, RR, TS)"); -- 2.32.0 From c3d8d04fae098c344a614eb67fbf1842dbd9444e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 25 Mar 2016 23:25:33 +0100 Subject: [PATCH 126/325] Detach freebsd-version from the build Obtained from: ElectroBSD --- bin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/bin/Makefile b/bin/Makefile index 3ad97ac8d624..d85e8019bd0e 100644 --- a/bin/Makefile +++ b/bin/Makefile @@ -15,7 +15,6 @@ SUBDIR= cat \ echo \ ed \ expr \ - freebsd-version \ getfacl \ hostname \ kenv \ -- 2.32.0 From da1f73a6a54b1fdadc74588dfd079940acba3f27 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 15:14:25 +0200 Subject: [PATCH 127/325] geli.8: Document that setkey option -P can't be combined with -i Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index b7bac3aecab6..ebe7824fb4c9 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -596,6 +596,8 @@ option. .It Fl P Do not use a passphrase as a component of the new User Key. Cannot be combined with the +.Fl i +or .Fl J option. .El -- 2.32.0 From d22fb6fd9c6a5b304f2816cde640d79a0ec51369 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 15:19:53 +0200 Subject: [PATCH 128/325] geli setkey: Reject combination of -P and -i Iterations aren't used for keys without passphrase and storing a iteration count would result in a passphrase promt that can't be used to attach the provider. Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index a89ac69cf338..28b68fa9a459 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -1397,6 +1397,8 @@ eli_setkey(struct gctl_req *req) struct g_eli_metadata md; const char *prov; int nargs; + intmax_t iterations; + int nonewpassphrase; nargs = gctl_get_int(req, "nargs"); if (nargs != 1) { @@ -1405,6 +1407,14 @@ eli_setkey(struct gctl_req *req) } prov = gctl_get_ascii(req, "arg0"); + nonewpassphrase = gctl_get_int(req, "nonewpassphrase"); + iterations = gctl_get_intmax(req, "iterations"); + if (iterations != -1 && nonewpassphrase) { + gctl_error(req, + "Options -i and -P are mutually exclusive."); + return; + } + if (eli_metadata_read(req, prov, &md) == -1) return; -- 2.32.0 From eb5c29ef22de5122adcb21ccc570f131afb6bbf8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 15:24:02 +0200 Subject: [PATCH 129/325] geli setkey: Prevent passphrase removal if two keys are configured The iterations are shared by both keys and resetting it for one would break the other one as a side-effect. If this is the intention the other key can still be removed with delkey before using setkey for the remaining one. Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index 28b68fa9a459..f01549a9c7a6 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -1418,6 +1418,14 @@ eli_setkey(struct gctl_req *req) if (eli_metadata_read(req, prov, &md) == -1) return; + if (nonewpassphrase && bitcount32(md.md_keys) != 1 && + md.md_iterations != -1) { + gctl_error(req, "To be able to switch from passphrase" + "-based key to passphrase-less key, only one can " + "be defined."); + return; + } + if (eli_is_attached(prov)) eli_setkey_attached(req, &md); else -- 2.32.0 From b475a264ed87b9946a7fd5777c53eb833ef64f8e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 15:37:03 +0200 Subject: [PATCH 130/325] geli.8: Document that setkey -P can't be used if two user keys with passphrase exist Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index ebe7824fb4c9..31bb5e386554 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -595,6 +595,9 @@ Cannot be combined with the option. .It Fl P Do not use a passphrase as a component of the new User Key. +This option cannot be used with the +.Cm setkey +subcommand if two User Keys with passphrase are configured. Cannot be combined with the .Fl i or -- 2.32.0 From 69642ca88f523f3c506e58b15e153868a1a12cb7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Mar 2016 14:22:52 +0200 Subject: [PATCH 131/325] geli setkey: Fix key change from passphrase-based to passphrase-less Previously the iterations count wasn't reset resulting in a passphrase prompt that would not accept any passphrase: fk@r500 ~ $sudo geli attach -k /bpool/boot/rpool.key /dev/gpt/rpool-ada1 Enter passphrase: fk@r500 ~ $geli dump /dev/gpt/rpool-ada1 Metadata on /dev/gpt/rpool-ada1: magic: GEOM::ELI version: 7 flags: 0x0 ealgo: AES-XTS keylen: 128 provsize: 1073741824 sectorsize: 512 keys: 0x01 iterations: 463852 Salt: [...] Master Key: [...] MD5 hash: e8693274fecc65d2a13c0071fb9413b3 fk@r500 ~ $sudo geli setkey -K /bpool/boot/rpool.key -P /dev/gpt/rpool-ada1 Note, that the master key encrypted with old keys and/or passphrase may still exists in a metadata backup file. fk@r500 ~ $geli dump /dev/gpt/rpool-ada1 Metadata on /dev/gpt/rpool-ada1: magic: GEOM::ELI version: 7 flags: 0x0 ealgo: AES-XTS keylen: 128 provsize: 1073741824 sectorsize: 512 keys: 0x01 iterations: 463852 Salt: [...] Master Key: [...] MD5 hash: a443402c3b97cb37494283f8f722994d fk@r500 ~ $sudo geli detach gpt/rpool-ada1 fk@r500 ~ $sudo geli attach -k /bpool/boot/rpool.key /dev/gpt/rpool-ada1 Enter passphrase: geli: Wrong key for gpt/rpool-ada1. From userland the promt could be suppressed with "-p" (which is required for passphrase-less keys anyway), but attaching at boot time wasn't possible. PR: 196834 Reported by: Julian Hsiao Obtained from: ElectroBSD --- lib/geom/eli/geom_eli.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index f01549a9c7a6..bd1c60c40c96 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -1273,8 +1273,16 @@ eli_setkey_attached(struct gctl_req *req, struct g_eli_metadata *md) /* Check if iterations number should be changed. */ if (val != -1) md->md_iterations = val; - else + else { old = md->md_iterations; + /* + * If the new key does not require a passphrase, + * the iterations count has to be reset to reflect + * this. + */ + if (gctl_get_int(req, "nonewpassphrase") == 1) + md->md_iterations = -1; + } /* Generate key for Master Key encryption. */ if (eli_genkey(req, md, key, true) == NULL) { @@ -1361,7 +1369,14 @@ eli_setkey_detached(struct gctl_req *req, const char *prov, return; } md->md_iterations = val; - } + } else if (gctl_get_int(req, "nonewpassphrase") == 1) { + /* + * If the new key does not require a passphrase, + * the iterations count has to be reset to reflect + * this. + */ + md->md_iterations = -1; + } mkeydst = md->md_mkeys + nkey * G_ELI_MKEYLEN; md->md_keys |= (1 << nkey); -- 2.32.0 From db5cc2a35988ce2949023aed481d00b98bcdcfe3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 4 Apr 2016 17:37:05 +0200 Subject: [PATCH 132/325] geli.8: Document the fact that both User Keys share an iteration value ... more explicitly. Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index 31bb5e386554..23bad1999806 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -812,6 +812,13 @@ the kernel from the Master Key and cached in memory. The number of Data Keys used by a given provider, and the way they are derived, depend on the GELI version and whether the provider is configured to use data authentication. +.Sh USER KEY LIMITATION +If the first User Key uses a passphrase, the second one has to use +a passphrase as well. +If the first User Key does not use a passphrase, the second one can +not use a passphrase either. +This limitation comes from the metadata format on disk which +currently only stores one iteration count for both keys. .Sh SYSCTL VARIABLES The following .Xr sysctl 8 -- 2.32.0 From 9567e13ef347ebe9b1c2d20d727e94f8818f8418 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 11 Apr 2016 11:45:44 +0200 Subject: [PATCH 133/325] etc/rc.d: Remove a couple of requirements that don't apply to ElectroBSD Obtained from: ElectroBSD --- libexec/rc/rc.d/NETWORKING | 4 ++-- libexec/rc/rc.d/bridge | 2 +- libexec/rc/rc.d/ipfw | 1 - libexec/rc/rc.d/ipnat | 28 ---------------------------- libexec/rc/rc.d/netif | 6 +++--- libexec/rc/rc.d/routing | 2 +- libexec/rc/rc.d/securelevel | 2 +- 7 files changed, 8 insertions(+), 37 deletions(-) delete mode 100755 libexec/rc/rc.d/ipnat diff --git a/libexec/rc/rc.d/NETWORKING b/libexec/rc/rc.d/NETWORKING index 9cdb5577ed2b..1bbea5aa40b9 100755 --- a/libexec/rc/rc.d/NETWORKING +++ b/libexec/rc/rc.d/NETWORKING @@ -4,8 +4,8 @@ # # PROVIDE: NETWORKING NETWORK -# REQUIRE: netif netwait netoptions routing ppp ipfw stf -# REQUIRE: defaultroute route6d resolv bridge +# REQUIRE: netif netwait netoptions routing ipfw stf +# REQUIRE: defaultroute routed route6d resolv bridge # REQUIRE: static_arp static_ndp # This is a dummy dependency, for services which require networking diff --git a/libexec/rc/rc.d/bridge b/libexec/rc/rc.d/bridge index 95e4eb9c2fac..8631fdab2b05 100755 --- a/libexec/rc/rc.d/bridge +++ b/libexec/rc/rc.d/bridge @@ -26,7 +26,7 @@ # # PROVIDE: bridge -# REQUIRE: netif ppp stf +# REQUIRE: netif stf # KEYWORD: nojail . /etc/rc.subr diff --git a/libexec/rc/rc.d/ipfw b/libexec/rc/rc.d/ipfw index 5d0bcc816560..69f034987c07 100755 --- a/libexec/rc/rc.d/ipfw +++ b/libexec/rc/rc.d/ipfw @@ -4,7 +4,6 @@ # # PROVIDE: ipfw -# REQUIRE: ppp # KEYWORD: nojailvnet . /etc/rc.subr diff --git a/libexec/rc/rc.d/ipnat b/libexec/rc/rc.d/ipnat deleted file mode 100755 index d4fa6b65dff4..000000000000 --- a/libexec/rc/rc.d/ipnat +++ /dev/null @@ -1,28 +0,0 @@ -#!/bin/sh -# -# $FreeBSD$ -# - -# PROVIDE: ipnat -# KEYWORD: nojailvnet - -. /etc/rc.subr - -name="ipnat" -desc="user interface to the NAT subsystem" -rcvar="ipnat_enable" -load_rc_config $name -start_cmd="ipnat_start" -stop_cmd="${ipnat_program} -F -C" -reload_cmd="${ipnat_program} -F -C -f ${ipnat_rules}" -extra_commands="reload" -required_files="${ipnat_rules}" -required_modules="ipl:ipfilter" - -ipnat_start() -{ - echo "Installing NAT rules." - ${ipnat_program} -CF -f ${ipnat_rules} ${ipnat_flags} -} - -run_rc_command "$1" diff --git a/libexec/rc/rc.d/netif b/libexec/rc/rc.d/netif index ff6962a1198e..bf7bfcee1353 100755 --- a/libexec/rc/rc.d/netif +++ b/libexec/rc/rc.d/netif @@ -26,9 +26,9 @@ # # PROVIDE: netif -# REQUIRE: FILESYSTEMS iovctl serial sppp sysctl -# REQUIRE: hostid ipfs -# KEYWORD: nojailvnet +# REQUIRE: FILESYSTEMS iovctl serial sysctl +# REQUIRE: hostid +# KEYWORD: nojailvnet shutdown . /etc/rc.subr . /etc/network.subr diff --git a/libexec/rc/rc.d/routing b/libexec/rc/rc.d/routing index e909523ab709..f65de7be30d2 100755 --- a/libexec/rc/rc.d/routing +++ b/libexec/rc/rc.d/routing @@ -6,7 +6,7 @@ # # PROVIDE: routing -# REQUIRE: netif ppp stf +# REQUIRE: netif stf # KEYWORD: nojailvnet . /etc/rc.subr diff --git a/libexec/rc/rc.d/securelevel b/libexec/rc/rc.d/securelevel index 8bb09dd920bd..f25841afcffd 100755 --- a/libexec/rc/rc.d/securelevel +++ b/libexec/rc/rc.d/securelevel @@ -4,7 +4,7 @@ # # PROVIDE: securelevel -# REQUIRE: adjkerntz ipfw pf sysctl_lastload +# REQUIRE: adjkerntz pf sysctl_lastload . /etc/rc.subr -- 2.32.0 From 6200570c82052c83b20f5e4fd24af90453f35af0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 25 Apr 2016 12:48:33 +0200 Subject: [PATCH 134/325] rc.d: Change geli_autodetach default to 'NO' ... as autodetach leads to panics when scrubbing ZFS pools with more than one device. For details see: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=117158 Obtained from: ElectroBSD --- libexec/rc/rc.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) mode change 100644 => 100755 libexec/rc/rc.conf diff --git a/libexec/rc/rc.conf b/libexec/rc/rc.conf old mode 100644 new mode 100755 index bd2fd5eeae2a..e5a731d09e21 --- a/libexec/rc/rc.conf +++ b/libexec/rc/rc.conf @@ -85,7 +85,7 @@ geli_groups="" # List of groups containing devices to automatically geli_tries="" # Number of times to attempt attaching geli device. # If empty, kern.geom.eli.tries will be used. geli_default_flags="" # Default flags for geli(8). -geli_autodetach="YES" # Automatically detach on last close. +geli_autodetach="NO" # Automatically detach on last close. # Providers are marked as such when all file systems are # mounted. # Example use. -- 2.32.0 From e9db60f516ef250291ad7d08f4a28e417d15c736 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 14 May 2016 22:05:48 +0200 Subject: [PATCH 135/325] disklatency: Skip invalid io::done probes .. until the cause has been analyzed and fixed Obtained from: ElectroBSD --- share/dtrace/disklatency | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/dtrace/disklatency b/share/dtrace/disklatency index 97ef87a07980..21ff6f719053 100755 --- a/share/dtrace/disklatency +++ b/share/dtrace/disklatency @@ -42,7 +42,7 @@ io:::start } io:::done -/this->start = start_time[arg0]/ +/(this->start = start_time[arg0]) && (args[1]->unit_number != -1)/ { this->delta = (timestamp - this->start) / 1000; @q[args[1]->device_name, args[1]->unit_number] = -- 2.32.0 From 0ba39113b0ba9d81d2f83bf39e278ae97458be9a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 14 May 2016 22:06:51 +0200 Subject: [PATCH 136/325] disklatency: Reformat output to print device name and unit together While at it, add a delimiter. Obtained from: ElectroBSD --- share/dtrace/disklatency | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/share/dtrace/disklatency b/share/dtrace/disklatency index 21ff6f719053..930be21d0afb 100755 --- a/share/dtrace/disklatency +++ b/share/dtrace/disklatency @@ -55,10 +55,11 @@ io:::done tick-10s { - printa(" %s (%d), us:\n%@d\n", @q); - printa("max%s (%d), us:\n%@d\n", @max); - printa("avg%s (%d), us:\n%@d\n", @avg); - printa("stddev%s (%d), us:\n%@d\n", @stddev); + printf("---------------------------------------------\n"); + printa("%s%d, us:\n%@d\n", @q); + printa("%s%d max, us:\n%@d\n", @max); + printa("%s%d avg, us:\n%@d\n", @avg); + printa("%s%d stdev, us:\n%@d\n", @stddev); clear(@q); clear(@max); -- 2.32.0 From 94f09d3500d7c9571e34d043df1d1eb1658470aa Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 15 May 2016 09:30:31 +0200 Subject: [PATCH 137/325] disklatency: Use trunc() instead of clear() ... so only latency stats for disks with activity in the relevant intervals are shown. Obtained from: ElectroBSD --- share/dtrace/disklatency | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/share/dtrace/disklatency b/share/dtrace/disklatency index 930be21d0afb..f4081c14bc84 100755 --- a/share/dtrace/disklatency +++ b/share/dtrace/disklatency @@ -61,8 +61,8 @@ tick-10s printa("%s%d avg, us:\n%@d\n", @avg); printa("%s%d stdev, us:\n%@d\n", @stddev); - clear(@q); - clear(@max); - clear(@avg); - clear(@stddev); + trunc(@q); + trunc(@max); + trunc(@avg); + trunc(@stddev); } -- 2.32.0 From afd75e54e3242919cc936746ae6fe5d94ff8faa0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 1 May 2016 16:27:55 +0200 Subject: [PATCH 138/325] vmstat: Prevent output truncation when piping zone statistics ... into other programs. Apparently libxo (currently?) can't be trusted to flush its internal buffer before it's full, triggering the flushing manually works around this. Before (sometimes): fk@r500 ~ $vmstat -z | wc -c 8192 After (consistently): fk@r500 ~ $vmstat -z | wc -c 24156 Obtained from: ElectroBSD PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206128 --- usr.bin/vmstat/vmstat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/usr.bin/vmstat/vmstat.c b/usr.bin/vmstat/vmstat.c index e8cec07fd46b..8632b218a966 100644 --- a/usr.bin/vmstat/vmstat.c +++ b/usr.bin/vmstat/vmstat.c @@ -1505,6 +1505,7 @@ domemstat_zone(void) (uintmax_t)memstat_get_failures(mtp), (uintmax_t)memstat_get_sleeps(mtp)); xo_close_instance("zone"); + xo_flush(); } memstat_mtl_free(mtlp); xo_close_list("zone"); -- 2.32.0 From b4af97d1214bd1b3647b4613463ec0c38b235998 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 27 May 2016 18:55:19 +0200 Subject: [PATCH 139/325] atkbdc: Work around (apparently) broken mouse reinitialization ... by increasing pkterrthresh to 10. Previously moving the mouse while loading DTrace modules, for example by running a DTrace script without having loaded the modules manually first, would reliably disables the mouse (track point) until reboot (on a ThinkPad R500). With debug.psm.loglevel=5 the logs showed: Nov 25 13:38:06 r500 kernel: [132] psmintr: 28 03 fe 01 ff 00 Nov 25 13:38:06 r500 kernel: [132] psmintr: 28 02 ff 01 ff 00 Nov 25 13:38:06 r500 kernel: [132] psmintr: 28 03 fe 02 ff 00 Nov 25 13:38:06 r500 kernel: [132] psmintr: 08 02 00 03 fe 00 Nov 25 13:38:06 r500 kernel: [132] psmintr: 28 01 ff 03 fe 00 Nov 25 13:38:07 r500 kernel: [132] psmintr: 18 fe 00 03 ff 00 Nov 25 13:38:08 r500 sudo: fk : TTY=ttyv0 ; PWD=/home/fk ; USER=root ; COMMAND=/usr/share/dtrace/toolkit/execsnoop Nov 25 13:38:08 r500 kernel: [134] psm0: lost interrupt? Nov 25 13:38:09 r500 kernel: [134] psmintr: 08 00 01 03 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 08 00 01 04 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 ff 01 04 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 ff 02 05 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 ff 02 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fe 03 05 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fe 04 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fd 05 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fd 05 03 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fe 03 05 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fd 03 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fa 06 05 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fc 04 06 ff 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fa 03 04 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 f9 05 07 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 f7 03 05 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 f9 04 06 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 f8 02 04 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fb 01 04 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 fb 01 02 00 00 Nov 25 13:38:09 r500 kernel: [134] psmintr: 18 ff 00 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 18 ff 00 02 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: c8 d3 0b 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: out of sync (00c0 != 0000) 881 cmds since last error. Nov 25 13:38:11 r500 kernel: [136] psmintr: discard a byte (1) Nov 25 13:38:11 r500 kernel: [136] psmintr: d3 0b 08 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: out of sync (00c0 != 0000) 0 cmds since last error. Nov 25 13:38:11 r500 kernel: [136] psmintr: discard a byte (2) Nov 25 13:38:11 r500 kernel: [136] psmintr: 0b 08 4b 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 3c 08 0c 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0f 08 0c 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 10 08 0b 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 10 08 0a 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 12 08 08 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 10 08 08 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 12 08 08 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 13 08 05 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0f 08 05 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 13 08 04 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 13 08 01 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0b 08 00 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0a 18 ff 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0f 18 fd 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0e 18 fc 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 08 18 fc 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 0a 18 fc 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 05 18 fb 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 05 18 fa 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 05 18 f5 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 09 18 f2 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 08 18 f2 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 05 18 f0 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 06 18 f2 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 03 18 ee 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 02 18 f3 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 01 18 ef 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: 00 38 ed 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: ff 38 ee 03 00 00 Nov 25 13:38:11 r500 kernel: [136] psmintr: out of sync (00c0 != 0000) 0 cmds since last error. Nov 25 13:38:11 r500 kernel: [136] psmintr: reset the mouse. Nov 25 13:38:12 r500 kernel: [137] psm0: current command byte: 0047 (reinitialize). Nov 25 13:38:12 r500 kernel: [137] psm: DISABLE_DEV return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: ENABLE_DEV return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: DISABLE_DEV return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: SET_SAMPLING_RATE (100) 00fa Nov 25 13:38:12 r500 kernel: [137] psm: SET_RESOLUTION (2) 00fa Nov 25 13:38:12 r500 kernel: [137] psm: SET_SCALING11 return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: SET_STREAM_MODE return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: SEND_AUX_DEV_STATUS return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: status 00 02 64 Nov 25 13:38:12 r500 kernel: [137] psm: ENABLE_DEV return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: SEND_AUX_DEV_STATUS return code:00fa Nov 25 13:38:12 r500 kernel: [137] psm: status 20 02 64 Nov 25 13:38:15 r500 kernel: [140] psm0: lost interrupt? Nov 25 13:38:16 r500 kernel: [141] psm0: lost interrupt? Nov 25 13:38:17 r500 kernel: [142] psm0: lost interrupt? Nov 25 13:38:18 r500 kernel: [143] psm0: lost interrupt? Nov 25 13:38:19 r500 kernel: [144] psm0: lost interrupt? Nov 25 13:38:20 r500 kernel: [145] psm0: lost interrupt? Nov 25 13:38:21 r500 kernel: [147] psm0: lost interrupt? Nov 25 13:38:22 r500 kernel: [148] psm0: lost interrupt? Nov 25 13:38:23 r500 kernel: [149] psm0: lost interrupt? Nov 25 13:38:24 r500 kernel: [150] psm0: lost interrupt? Nov 25 13:38:25 r500 kernel: [151] psm0: lost interrupt? Nov 25 13:38:26 r500 kernel: [152] psm0: lost interrupt? Nov 25 13:38:27 r500 kernel: [153] psm0: lost interrupt? Nov 25 13:38:29 r500 kernel: [154] psm0: lost interrupt? Nov 25 13:38:30 r500 kernel: [155] psm0: lost interrupt? Nov 25 13:38:31 r500 kernel: [156] psm0: lost interrupt? Nov 25 13:38:32 r500 kernel: [157] psm0: lost interrupt? Nov 25 13:38:33 r500 kernel: [158] psm0: lost interrupt? Nov 25 13:38:34 r500 kernel: [159] psm0: lost interrupt? Nov 25 13:38:35 r500 kernel: [160] psm0: lost interrupt? Nov 25 13:38:36 r500 kernel: [161] psm0: lost interrupt? Nov 25 13:38:37 r500 kernel: [162] psm0: lost interrupt? Nov 25 13:38:38 r500 kernel: [163] psm0: lost interrupt? Nov 25 13:38:39 r500 kernel: [164] psm0: lost interrupt? Nov 25 13:38:40 r500 kernel: [165] psm0: lost interrupt? After the "reset" the mouse cursor no longer moved and rebooting seemed to be the only "cure". Setting debug.psm.pkterrthresh=10 seems to work around the issue, so does not moving the cursor until the modules are loaded. Mouse resets for other reasons seemed to occasionally render the mouse useless as well, but triggering the problem by loading DTrace modules is the only reproducible method currently known. Obtained from: ElectroBSD PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221305 PR submission date: 2017-08-07 --- sys/dev/atkbdc/psm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/atkbdc/psm.c b/sys/dev/atkbdc/psm.c index cae4ac41fb32..de9705efd160 100644 --- a/sys/dev/atkbdc/psm.c +++ b/sys/dev/atkbdc/psm.c @@ -2966,7 +2966,7 @@ SYSCTL_INT(_debug_psm, OID_AUTO, secs, CTLFLAG_RW, &psmsecs, 0, static int psmusecs = 500000; SYSCTL_INT(_debug_psm, OID_AUTO, usecs, CTLFLAG_RW, &psmusecs, 0, "Microseconds to add to psmsecs"); -static int pkterrthresh = 2; +static int pkterrthresh = 10; SYSCTL_INT(_debug_psm, OID_AUTO, pkterrthresh, CTLFLAG_RW, &pkterrthresh, 0, "Number of error packets allowed before reinitializing the mouse"); -- 2.32.0 From 77243473ca053a119750bba2bd4d3bd8f011f123 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 24 Jun 2016 14:39:28 +0200 Subject: [PATCH 140/325] sys/amd64/conf/ELECTRO_BLOAT: Remove COMPAT support for FreeBSD versions before 10 Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 5 ----- 1 file changed, 5 deletions(-) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index a61f4e5eee3a..3bd60d2df610 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -48,11 +48,6 @@ options GEOM_PART_GPT # GUID Partition Tables. options GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization options COMPAT_FREEBSD32 # Compatible with i386 binaries -options COMPAT_FREEBSD4 # Compatible with FreeBSD4 -options COMPAT_FREEBSD5 # Compatible with FreeBSD5 -options COMPAT_FREEBSD6 # Compatible with FreeBSD6 -options COMPAT_FREEBSD7 # Compatible with FreeBSD7 -options COMPAT_FREEBSD9 # Compatible with FreeBSD9 options COMPAT_FREEBSD10 # Compatible with FreeBSD10 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support -- 2.32.0 From aae28b742c2d456800fcf456c41e8aae711eaadd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 13 Feb 2015 17:56:07 +0100 Subject: [PATCH 141/325] range_tree_destroy(): Optionally tolerate non-zero rt->rt_space It's unclear if this is still needed but keeping it around doesn't hurt. Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c index fc705e37964d..fb71f6090fbe 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c @@ -197,7 +197,11 @@ range_tree_create(range_tree_ops_t *ops, void *arg) void range_tree_destroy(range_tree_t *rt) { - VERIFY0(rt->rt_space); + if (rt->rt_space != 0) { + zfs_panic_recover("zfs: range_tree_destroy(): " + "rt->rt_space != 0: %llx", rt->rt_space); + return; + } if (rt->rt_ops != NULL && rt->rt_ops->rtop_destroy != NULL) rt->rt_ops->rtop_destroy(rt, rt->rt_arg); -- 2.32.0 From 7c310abcf23002c86d0bd19bdb3c82e1460c1623 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 5 May 2016 13:59:47 +0200 Subject: [PATCH 142/325] share/skel: Detach skelleton files for csh and rsh from the build Obtained from: ElectroBSD --- share/skel/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/skel/Makefile b/share/skel/Makefile index 9ad66cff61e8..784a9adeaf84 100644 --- a/share/skel/Makefile +++ b/share/skel/Makefile @@ -1,7 +1,7 @@ # @(#)Makefile 8.1 (Berkeley) 6/8/93 # $FreeBSD$ -FILES= dot.cshrc dot.login dot.login_conf dot.mailrc dot.profile \ +FILES= dot.login dot.login_conf dot.mailrc dot.profile \ dot.shrc dot.mail_aliases FILESDIR= ${SHAREDIR}/skel FILESMODE= 0644 -- 2.32.0 From a3ba1d2bd22ad6f00d60c39cfe87eb9be1de1761 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 20 Sep 2015 19:53:09 +0200 Subject: [PATCH 143/325] top: Show ZFS ARC target size Obtained from: ElectroBSD --- usr.bin/top/machine.c | 4 +++- usr.bin/top/top.1 | 8 +++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/usr.bin/top/machine.c b/usr.bin/top/machine.c index 6116a06bb28b..cd1c61fe6b8a 100644 --- a/usr.bin/top/machine.c +++ b/usr.bin/top/machine.c @@ -133,7 +133,7 @@ static const char *memorynames[] = { static int memory_stats[nitems(memorynames)]; static const char *arcnames[] = { - "K Total, ", "K MFU, ", "K MRU, ", "K Anon, ", "K Header, ", "K Other", + "K Total, ", "K MFU, ", "K MRU, ", "K Anon, ", "K Header, ", "K Other, ", "K Target", NULL }; static int arc_stats[nitems(arcnames)]; @@ -535,6 +535,8 @@ get_system_info(struct system_info *si) arc_stats[5] += arc_stat >> 10; GETSYSCTL("kstat.zfs.misc.arcstats.dbuf_size", arc_stat); arc_stats[5] += arc_stat >> 10; + GETSYSCTL("kstat.zfs.misc.arcstats.c", arc_stat); + arc_stats[6] = arc_stat >> 10; si->arc = arc_stats; } if (carc_enabled) { diff --git a/usr.bin/top/top.1 b/usr.bin/top/top.1 index 4a587877dba6..87334a1a7792 100644 --- a/usr.bin/top/top.1 +++ b/usr.bin/top/top.1 @@ -442,7 +442,7 @@ are not. .Sh DESCRIPTION OF MEMORY .Bd -literal Mem: 61M Active, 86M Inact, 368K Laundry, 22G Wired, 102G Free -ARC: 15G Total, 9303M MFU, 6155M MRU, 1464K Anon, 98M Header, 35M Other +ARC: 15G Total, 9303M MFU, 6155M MRU, 1464K Anon, 98M Header, 35M Other, 15G Target 15G Compressed, 27G Uncompressed, 1.75:1 Ratio, 174M Overhead Swap: 4096M Total, 532M Free, 13% Inuse, 80K In, 104K Out .Ed @@ -477,6 +477,12 @@ number of ARC bytes holding in flight data number of ARC bytes holding headers .It Em Other miscellaneous ARC bytes +.B Target: +ARC target size, that is the total amount of memory +the ARC considers usable for itself. If it's not equal +to the total size, the ARC will shrink or grow to reach +the target. +.TP .It Em Compressed bytes of memory used by ARC caches .It Em Uncompressed -- 2.32.0 From 02120279b26fe13703e6582ad11f7a7a4fc81ad7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 25 Oct 2015 19:49:47 +0100 Subject: [PATCH 144/325] ZFS dsl_scan_visitds(): Don't panic if a device disappears while scrubbing Prevents: Unread portion of the kernel message buffer: [4299] panic: solaris assert: dmu_objset_find_dp(dp, dp->dp_root_dir_obj, enqueue_clones_cb, &eca, (1<<1)) == 0 (0x6 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c, line: 1130 [4299] cpuid = 1 [4299] KDB: stack backtrace: [4299] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00949ed310 [4299] vpanic() at vpanic+0x182/frame 0xfffffe00949ed390 [4299] panic() at panic+0x43/frame 0xfffffe00949ed3f0 [4299] zfs_kmem_alloc() at zfs_kmem_alloc/frame 0xfffffe00949ed440 [4299] dsl_scan_visitds() at dsl_scan_visitds+0x551/frame 0xfffffe00949ed570 [4299] dsl_scan_visit() at dsl_scan_visit+0x22e/frame 0xfffffe00949ed790 [4299] dsl_scan_sync() at dsl_scan_sync+0x9da/frame 0xfffffe00949ed920 [4299] spa_sync() at spa_sync+0x564/frame 0xfffffe00949eda90 [4299] txg_sync_thread() at txg_sync_thread+0x3f1/frame 0xfffffe00949edbb0 [4299] fork_exit() at fork_exit+0x9c/frame 0xfffffe00949edbf0 [4299] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00949edbf0 [4299] --- trap 0, rip = 0, rsp = 0, rbp = 0 --- [4299] KDB: enter: panic Note that this is not the only place where dmu_objset_find_dp() failures lead to panics and ENXIO probably isn't the only return code we have to expect anyway. Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c index 48c0f7a593a0..554878b7188e 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c @@ -2293,9 +2293,11 @@ dsl_scan_visitds(dsl_scan_t *scn, uint64_t dsobj, dmu_tx_t *tx) } zap_cursor_fini(&zc); } else { - VERIFY0(dmu_objset_find_dp(dp, dp->dp_root_dir_obj, + int error; + error = dmu_objset_find_dp(dp, dp->dp_root_dir_obj, enqueue_clones_cb, &ds->ds_object, - DS_FIND_CHILDREN)); + DS_FIND_CHILDREN); + VERIFY(error == 0 || error == ENXIO); } } -- 2.32.0 From e7e57fbb5afe49883b3645d1b4adb650d989ae65 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 30 Dec 2016 16:54:28 +0100 Subject: [PATCH 145/325] g_dev_orphan(): Return early if the device is already gone Supposed to fix a panic that could occur while running "cdcontrol eject" after using the physical ejection key on the device: Unread portion of the kernel message buffer: stack pointer = 0x28:0xfffffe01eba0a9e0 frame pointer = 0x28:0xfffffe01eba0a9f0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 13 (g_event) trap number = 12 panic: page fault cpuid = 3 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01eba0a560 vpanic() at vpanic+0x182/frame 0xfffffe01eba0a5e0 panic() at panic+0x43/frame 0xfffffe01eba0a640 trap_fatal() at trap_fatal+0x331/frame 0xfffffe01eba0a6a0 trap_pfault() at trap_pfault+0x1e3/frame 0xfffffe01eba0a700 trap() at trap+0x273/frame 0xfffffe01eba0a910 calltrap() at calltrap+0x8/frame 0xfffffe01eba0a910 --- trap 0xc, rip = 0xffffffff80500fde, rsp = 0xfffffe01eba0a9e0, rbp = 0xfffffe01eba0a9f0 --- g_dev_orphan() at g_dev_orphan+0x2e/frame 0xfffffe01eba0a9f0 g_resize_provider_event() at g_resize_provider_event+0x71/frame 0xfffffe01eba0aa20 g_run_events() at g_run_events+0x20e/frame 0xfffffe01eba0aa70 fork_exit() at fork_exit+0x85/frame 0xfffffe01eba0aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01eba0aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 3h17m41s Dumping 1120 out of 8055 MB:..2%..12%..22%..32%..42%..52%..62%..72%..82%..92% Reading symbols from /usr/lib/debug/boot/kernel/zfs.ko.debug...done. [...] Loaded symbols for /usr/lib/debug/boot/kernel/iicbb.ko.debug #0 doadump (textdump=1) at pcpu.h:222 222 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump (textdump=1) at pcpu.h:222 #1 0xffffffff805cce3e in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff805cd40b in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff805cd243 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:690 #4 0xffffffff808eece1 in trap_fatal (frame=0xfffffe01eba0a920, eva=8) at /usr/src/sys/amd64/amd64/trap.c:801 #5 0xffffffff808eeed3 in trap_pfault (frame=0xfffffe01eba0a920, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:658 #6 0xffffffff808ee4d3 in trap (frame=0xfffffe01eba0a920) at /usr/src/sys/amd64/amd64/trap.c:421 #7 0xffffffff808d2701 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #8 0xffffffff80500fde in g_dev_orphan (cp=0xfffff800069e6280) at /usr/src/sys/geom/geom_dev.c:754 #9 0xffffffff80509ff1 in g_resize_provider_event (arg=, flag=) at /usr/src/sys/geom/geom_subr.c:631 #10 0xffffffff80504f1e in g_run_events () at /usr/src/sys/geom/geom_event.c:264 #11 0xffffffff805830e5 in fork_exit (callout=0xffffffff805079c0 , arg=0x0, frame=0xfffffe01eba0aac0) at /usr/src/sys/kern/kern_fork.c:1040 #12 0xffffffff808d2c3e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611 #13 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) f 8 #8 0xffffffff80500fde in g_dev_orphan (cp=0xfffff800069e6280) at /usr/src/sys/geom/geom_dev.c:754 754 g_trace(G_T_TOPOLOGY, "g_dev_orphan(%p(%s))", cp, cp->geom->name); (kgdb) p cp $1 = (struct g_consumer *) 0xfffff800069e6280 (kgdb) p cp->geom $2 = (struct g_geom *) 0xfffff801acd6d100 (kgdb) p cp->geom->name $3 = 0xfffff8007479bf60 "cd0" (kgdb) f 8 #8 0xffffffff80500fde in g_dev_orphan (cp=0xfffff800069e6280) at /usr/src/sys/geom/geom_dev.c:754 754 g_trace(G_T_TOPOLOGY, "g_dev_orphan(%p(%s))", cp, cp->geom->name); (kgdb) l 754 g_trace(G_T_TOPOLOGY, "g_dev_orphan(%p(%s))", cp, cp->geom->name); 755 756 /* Reset any dump-area set on this device */ 757 if (dev->si_flags & SI_DUMPDEV) 758 (void)set_dumper(NULL, NULL, curthread); 759 760 /* Destroy the struct cdev *so we get no more requests */ 761 destroy_dev_sched_cb(dev, g_dev_callback, cp); 762 } 763 (kgdb) p dev->si_flags Cannot access memory at address 0x8 (kgdb) p dev $4 = (struct cdev *) 0x0 Last message before the panic: Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:23 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:24 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:26 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c0 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 39 c6 c1 00 00 01 00 Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:6f,3 (Read of scrambled sector without authentication) Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:09:33 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:33:49 t520 kernel: ahcich1: Timeout on slot 8 port 0 Dec 30 16:33:49 t520 kernel: ahcich1: is 00000000 cs 00000100 ss 00000000 rs 00000100 tfd d0 serr 00000000 cmd 000cc817 Dec 30 16:33:49 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 eb 00 00 01 00 Dec 30 16:33:49 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: Command timeout Dec 30 16:33:49 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 eb 00 00 01 00 Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: MEDIUM ERROR asc:10,0 (ID CRC or ECC error) Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:33:57 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 ad 00 00 04 00 Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: MEDIUM ERROR asc:10,0 (ID CRC or ECC error) Dec 30 16:34:05 t520 kernel: (cd0:ahcich1:0:0:0): Retrying command (per sense data) Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 ad 00 00 04 00 Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: MEDIUM ERROR asc:10,0 (ID CRC or ECC error) Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): Error 5, Retries exhausted Dec 30 16:34:14 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x5 back Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 eb 00 00 01 00 Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): Error 6, Unretryable error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x6 back Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 ea 00 00 01 00 Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): Error 6, Unretryable error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x6 back Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 ea 00 00 01 00 Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): Error 6, Unretryable error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x6 back Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): READ(10). CDB: 28 00 00 3e a2 eb 00 00 01 00 Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): CAM status: SCSI Status Error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI status: Check Condition Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): Error 6, Unretryable error Dec 30 16:34:18 t520 kernel: (cd0:ahcich1:0:0:0): cddone: got error 0x6 back PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215856 PR submission date: 2017-01-07 Obtained from: ElectroBSD --- sys/geom/geom_dev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sys/geom/geom_dev.c b/sys/geom/geom_dev.c index 1d20e4e0bf0e..e5e34dbb83d6 100644 --- a/sys/geom/geom_dev.c +++ b/sys/geom/geom_dev.c @@ -833,6 +833,9 @@ g_dev_orphan(struct g_consumer *cp) dev = sc->sc_dev; g_trace(G_T_TOPOLOGY, "g_dev_orphan(%p(%s))", cp, cp->geom->name); + if (dev == NULL) + return; + /* Reset any dump-area set on this device */ if (dev->si_flags & SI_DUMPDEV) (void)clear_dumper(curthread); -- 2.32.0 From 6aa8af28a1c97bda5229d5990dfb7dcdb7a2794d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 25 Jun 2015 17:26:59 +0200 Subject: [PATCH 146/325] Add reproduce.sh which makes reproducing ElectroBSD more convenient Squashed commits worth mentioning: - Add -j option to overwrite the maximum number of make jobs - Assert that the source directory is untainted (according to strip-freebsd.sh) and add -a flag to remove offending files - Allow to resume a build by using the -r flag. - Add -p option to change the prefix for the source and object directories This could be useful when building ElectroBSD as port. - Add a -d option to specifiy the directory to move the produced distfiles into Obtained from: ElectroBSD --- reproduce.sh | 250 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 250 insertions(+) create mode 100755 reproduce.sh diff --git a/reproduce.sh b/reproduce.sh new file mode 100755 index 000000000000..996212b07665 --- /dev/null +++ b/reproduce.sh @@ -0,0 +1,250 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2015-2016 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## + +# reproduce.sh +# +# Script to make reproducing an ElectroBSD build more convenient. +# Before using it, make sure BUILD and EPOCH contain values other +# than __BUILD__ and __EPOCH__, either by editing the script or +# by putting them in a configuration file that is speficied with +# the -f option. + +# These variables have to be set to the values used for the build +# that is supposed to be reproduced. ${SRC_DIR} must contain the +# matching sources! +BUILD=__BUILD__ +EPOCH=__EPOCH__ + +# This is just a suggestion, feel free to overwrite it with the -j option. +MAX_MAKE_JOBS="${MAX_MAKE_JOBS-2}" + +# Prefix to use for SRC_DIR and MAKEOBJDIRPREFIX +DIRECTORY_PREFIX="${DIRECTORY_PREFIX-/}" + +# Currently somewhat hardcoded. +SRC_DIR="${DIRECTORY_PREFIX}usr/src" +MAKEOBJDIRPREFIX="${DIRECTORY_PREFIX}usr/obj" + +# Config file location when -f isn't specified +OPTIONAL_CONFIG_FILE="${SRC_DIR}/reproduce.conf" + +# Make sure we respawn with the same script, even if it is located +# outside the SRC_DIR and called with a relative path. +REPRODUCE_SH="$(realpath "${0}")" + +# When set to true, existing object files will be reused. +# If the source files changed, the result will not be reproducible! +RESUME_BUILD="${RESUME_BUILD-false}" + +# Number of threads to use when compressing with xz. +# +# The upstream default is 0 (auto-tune) which results +# in unreproducible results when using systems with a +# different number of cores. +XZ_THREADS="${XZ_THREADS-2}" + +announce_status() { + local msg \ + timestamp + + msg="${*}" + timestamp=$(date "+%Y-%m-%d %H:%M") + + echo "${timestamp}: ${msg}" +} + +reproduce_all_the_things() { + local \ + f release_dir + + # In a perfect world the umask shouldn't matter here but + # in the real world it does for the single file /etc/rc.local + # on the memstick. + umask 022 + release_dir="${MAKEOBJDIRPREFIX}${SRC_DIR}/release" + if "${RESUME_BUILD}"; then + announce_status "Resuming ..." + export KERNFAST=1 + export NO_CLEAN=1 + fi + export MAKEOBJDIRPREFIX + + # We build the world first, so the kernel is built + # with a freshly built toolchain. + announce_status "Starting to build the world" + time -l make -j${MAX_MAKE_JOBS} buildworld || return 1 + + announce_status "Starting to build the kernel" + time -l make buildkernel || return 1 + + # Make sure obj files aren't dumped in ${SRC_DIR} + mkdir -p "${release_dir}" || return 1 + + if ! "${RESUME_BUILD}"; then + announce_status "Starting to clean the release dir" + make -C "${SRC_DIR}/release" clean + fi + announce_status "Starting to build the release" + time -l make -C "${SRC_DIR}/release" \ + memstick XZ_THREADS=${XZ_THREADS} NO_FSCHG="yes" || return 1 + + announce_status "Done with release memstick for ${REPRO_SEED}" + + cd "${release_dir}" || return 1 + + mkdir -p "${DISTFILE_DIR}" || return 1 + for f in *.txz MANIFEST; do + mv "${f}" "${DISTFILE_DIR}/" || return 1 + done + mv memstick.img "${DISTFILE_DIR}/${BUILD}.img" || return 1 + + announce_status "Created files copied to ${DISTFILE_DIR}" + sha256 "${DISTFILE_DIR}"/* +} + +assert_untainted_source_tree() { + local auto_untaint \ + untaint_commands + + auto_untaint="${1}" + + untaint_commands="$(sh ./release/scripts/strip-freebsd.sh .)" + if [ -n "${untaint_commands}" ] ; then + if $auto_untaint; then + echo "Auto untainting $(pwd)" + echo "${untaint_commands}" | sh -x || return 1 + else + echo "${SRC_DIR} is tainted. Use -a flag to auto-untaint it." + return 1 + fi + fi + return 0 +} + +respawn_with_clean_environment() { + exec env -i PATH="/sbin:/bin:/usr/sbin:/usr/bin" HOME="/root" \ + LC_COLLATE=C SHELL=/bin/sh ALREADY_RESPAWNED=1 \ + DISTFILE_DIR="${DISTFILE_DIR}" \ + MAX_MAKE_JOBS="${MAX_MAKE_JOBS}" RESUME_BUILD="${RESUME_BUILD}" \ + DIRECTORY_PREFIX="${DIRECTORY_PREFIX}" XZ_THREADS=${XZ_THREADS} \ + REPRO_SEED="${REPRO_SEED}" SRCCONF=/dev/null /bin/sh "${REPRODUCE_SH}" +} + +main() { + local args \ + auto_untaint config_file fake_user dry_run + + fake_user=elektropunker + + auto_untaint=false + dry_run=false + config_file="${OPTIONAL_CONFIG_FILE}" + + args=$(getopt ad:f:j:npr $*) + if [ $? -ne 0 ]; then + echo 'You are doing it wrong: Invalid flag specified' + exit 2 + fi + set -- ${args} + while true; do + case "$1" in + -a) + shift + auto_untaint="true" + ;; + -d) + shift; + DISTFILE_DIR="${1}" + shift; + ;; + -j) + shift; + MAX_MAKE_JOBS="${1}" + shift; + ;; + -f) + shift; + config_file="${1}" + shift; + if [ ! -f "${config_file}" ]; then + echo "Config file ${config_file} does not exist" + exit 2 + fi + ;; + -n) + dry_run=true + shift + ;; + -p) + shift + DIRECTORY_PREFIX="${1}" + shift; + if [ ! -d "${DIRECTORY_PREFIX}" ]; then + echo "Directory ${DIRECTORY_PREFIX} specified with -p does not exist" + exit 2 + fi + # This is only needed for the cd below + SRC_DIR="${DIRECTORY_PREFIX}${SRC_DIR}" + ;; + -r) + shift + RESUME_BUILD=true + ;; + --) + shift; break + ;; + esac + done + + if [ -f "${config_file}" ]; then + announce_status "Reading config from ${config_file}" + . "${config_file}" || exit 2 + fi + + if [ "${BUILD}" = "__BUILD__" ]; then + announce_status "BUILD not set" + return 1 + fi + if [ "${EPOCH}" = "__EPOCH__" ]; then + announce_status "EPOCH not set" + return 1 + fi + if [ -n "${ALREADY_RESPAWNED}" -a "${ALREADY_RESPAWNED}" = 1 ]; then + if [ -z "${REPRO_SEED}" ]; then + announce_status "Respawned with REPRO_SEED unset" + return 1 + fi + reproduce_all_the_things + return + fi + + export REPRO_SEED="${fake_user}:${BUILD}:${EPOCH}" + export DISTFILE_DIR="${DISTFILE_DIR-${MAKEOBJDIRPREFIX}${SRC_DIR}/${BUILD}-$(date +%Y-%m-%d-%H:%M)}" + + announce_status "REPRO_SEED=${REPRO_SEED}" + + cd "${SRC_DIR}" || return 1 + + assert_untainted_source_tree "${auto_untaint}" || return 1 + + if ! $dry_run; then + respawn_with_clean_environment + fi +} + +main "${@}" -- 2.32.0 From 643abf302db1fac21234d61f8f0a8e1753766ad2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 18 Apr 2015 19:38:07 +0200 Subject: [PATCH 147/325] gmountver.8: Note that GEOM mount verification has dangerous bugs Obtained from: ElectroBSD --- lib/geom/mountver/gmountver.8 | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/lib/geom/mountver/gmountver.8 b/lib/geom/mountver/gmountver.8 index 4c27a652b0b0..ddb9e2b2a73f 100644 --- a/lib/geom/mountver/gmountver.8 +++ b/lib/geom/mountver/gmountver.8 @@ -61,6 +61,8 @@ got disconnected - it queues all the I/O requests and waits for the provider to reappear. When that happens, it attaches to it and sends the queued requests. .Pp +At least that's the theory, please note the BUGS section. +.Pp The first argument to .Nm indicates an action to be performed: @@ -119,6 +121,13 @@ If set to 0, .Nm will reattach to the device even if the device reports a different disk ID. .El +.Sh BUGS +The mount verification GEOM class can stall all the disk I/O instead +of just the device it is configured for. +The disk identification check currently has to be turned off for the class +to actually attach reappearing providers. +Obviously this is dangerous. +.El .Sh EXIT STATUS Exit status is 0 on success, and 1 if the command fails. .Sh SEE ALSO -- 2.32.0 From de87cf793fd2509c9b5062dccdbec6e785c1bf63 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 28 Jan 2017 14:41:17 +0100 Subject: [PATCH 148/325] Prevent inlining of g_mountver_ident_matches() so the taste failures can be debugged Obtained from: ElectroBSD --- sys/geom/mountver/g_mountver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/geom/mountver/g_mountver.c b/sys/geom/mountver/g_mountver.c index 1088c98b179d..63d0d6dff758 100644 --- a/sys/geom/mountver/g_mountver.c +++ b/sys/geom/mountver/g_mountver.c @@ -526,7 +526,7 @@ g_mountver_resize(struct g_consumer *cp) g_resize_provider(pp, cp->provider->mediasize); } -static int +static __noinline int g_mountver_ident_matches(struct g_geom *gp) { struct g_consumer *cp; -- 2.32.0 From 2fb4e368e88b5228d26291d1912c701b5364bd97 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 28 Jan 2017 16:43:27 +0100 Subject: [PATCH 149/325] geom/mountver: Bump log level for G_MOUNTVER_LOGREQ to 3 to reduce noise when debugging tasting Obtained from: ElectroBSD --- sys/geom/mountver/g_mountver.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/geom/mountver/g_mountver.h b/sys/geom/mountver/g_mountver.h index 6d7c2d7241c5..360562c2dd8b 100644 --- a/sys/geom/mountver/g_mountver.h +++ b/sys/geom/mountver/g_mountver.h @@ -49,8 +49,8 @@ } \ } while (0) #define G_MOUNTVER_LOGREQ(bp, ...) do { \ - if (g_mountver_debug >= 2) { \ - printf("GEOM_MOUNTVER[2]: "); \ + if (g_mountver_debug >= 3) { \ + printf("GEOM_MOUNTVER[3]: "); \ printf(__VA_ARGS__); \ printf(" "); \ g_print_bio(bp); \ -- 2.32.0 From 8a3e74117a1d5b6ee47a3e0d7f02a4873a2fdb26 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 29 Jan 2017 14:49:10 +0100 Subject: [PATCH 150/325] release/rc.local: Let the memstick create a script to self-reproduce Only works if there's enough free space at the end of the device the image is located on. Before trying to reproduce in a virtualized environment like bhyve increase the memstick size with something like: truncate -s +20G [...]/ElectroBSD-r314503-7314f38a400e.img Obtained from: ElectroBSD --- release/rc.local | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/release/rc.local b/release/rc.local index 7acf5070ef5e..776ab73e9719 100755 --- a/release/rc.local +++ b/release/rc.local @@ -10,6 +10,35 @@ MACHINE=`uname -m` +# create a script that can be used to reproduce the image with itself +# (provided there is enough space at the end to create another partition) +cat << 'EOF' > /tmp/reproduce-from-image.sh +set -x -e +# Fingers crossed that the first disk is the boot disk ... +DISK=$(geom disk status -s | head -n 1 | cut -f 1 -w) +JAILROOT="$(mktemp -d /tmp/ElectroBSD-jail-XXXXXX)" +SWAP_PARTITION_INDEX=3 +BUILD_PARTITION_INDEX=4 + +gpart recover "${DISK}" +gpart add -t freebsd-swap -s 1500M "${DISK}" +swapon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +dumpon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +gpart add -t freebsd-ufs "${DISK}" +# gpart partition changes made in bhyve are currently +# lost. The cause hasn't been diagnosed yet. +# Explicitly include the partition layout in the +# presumably logged output so it can be restored +# afterwards. +gpart show "${DISK}" # More convenient format for humans +gpart backup "${DISK}" # Same data in format understood by "gpart restore" +newfs "/dev/${DISK}p${BUILD_PARTITION_INDEX}" +mount -o async,noatime "/dev/${DISK}p${BUILD_PARTITION_INDEX}" "${JAILROOT}" +tar xf /usr/electrobsd-dist/base.txz -C "${JAILROOT}" +tar xf /usr/electrobsd-dist/src.txz -C "${JAILROOT}" +time jail -c path="${JAILROOT}" mount.devfs host.hostname=ElectroBSD command=sh /usr/src/reproduce.sh +EOF + # resolv.conf from DHCP ends up in here, so make sure the directory exists mkdir /tmp/bsdinstall_etc -- 2.32.0 From cfd5bf337ced3058012051fc00e808a41221f5ae Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 25 Apr 2017 20:01:31 +0200 Subject: [PATCH 151/325] release/rc.local: Add /tmp/savecore-and-go-to-jail script Obtained from: ElectroBSD --- release/rc.local | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/release/rc.local b/release/rc.local index 776ab73e9719..a5977c3a8f84 100755 --- a/release/rc.local +++ b/release/rc.local @@ -39,6 +39,33 @@ tar xf /usr/electrobsd-dist/src.txz -C "${JAILROOT}" time jail -c path="${JAILROOT}" mount.devfs host.hostname=ElectroBSD command=sh /usr/src/reproduce.sh EOF +# create a script to recover a core dump and go to the jail to analyze it +cat << 'EOF' > /tmp/savecore-and-go-to-jail +set -x -e +# Fingers crossed that the first disk is the boot disk ... +DISK=$(geom disk status -s | head -n 1 | cut -f 1 -w) +JAILROOT="$(mktemp -d /tmp/ElectroBSD-jail-XXXXXX)" +SWAP_PARTITION_INDEX=3 +BUILD_PARTITION_INDEX=4 + +gpart recover "${DISK}" +gpart add -t freebsd-swap -s 1500M "${DISK}" +swapon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +dumpon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +gpart add -t freebsd-ufs "${DISK}" +# gpart partition changes made in bhyve are currently +# lost. The cause hasn't been diagnosed yet. +# Explicitly include the partition layout in the +# presumably logged output so it can be restored +# afterwards. +gpart show "${DISK}" # More convenient format for humans +gpart backup "${DISK}" # Same data in format understood by "gpart restore" +fsck -C -y "/dev/${DISK}p${BUILD_PARTITION_INDEX}" +mount -o async,noatime "/dev/${DISK}p${BUILD_PARTITION_INDEX}" "${JAILROOT}" +savecore -v "${JAILROOT}/var/crash" "/dev/${DISK}p${SWAP_PARTITION_INDEX}" +time jail -c path="${JAILROOT}" mount.devfs host.hostname=ElectroBSD command=sh +EOF + # resolv.conf from DHCP ends up in here, so make sure the directory exists mkdir /tmp/bsdinstall_etc -- 2.32.0 From 5c4f5770d0efb96bcd745797af6d924b7589ea93 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 30 Mar 2015 15:24:05 +0200 Subject: [PATCH 152/325] ggatel: Optionally retry in case of failed reads and writes ... after waiting five seconds and reopening the device. This allows to use ggatel to work around moderate USB flakiness which can result in device disconnects that are extremely annoying if ZFS on geli is involved. To prevent data loss if multiple device disappear at the same time and reappear with different names, the disk ident is checked to confirm that the device is the expected one. As a side-effect retrying will not work for file-backed memory disks (which currently have no disk ident). Obtained from: ElectroBSD --- sbin/ggate/ggatel/ggatel.c | 67 +++++++++++++++++++++++++++++++++++--- 1 file changed, 63 insertions(+), 4 deletions(-) diff --git a/sbin/ggate/ggatel/ggatel.c b/sbin/ggate/ggatel/ggatel.c index 4659cacbee17..88d0bf25ae46 100644 --- a/sbin/ggate/ggatel/ggatel.c +++ b/sbin/ggate/ggatel/ggatel.c @@ -55,6 +55,7 @@ static const char *path = NULL; static int unit = G_GATE_UNIT_AUTO; static unsigned flags = 0; static int force = 0; +static unsigned retries = 0; static unsigned sectorsize = 0; static unsigned timeout = G_GATE_TIMEOUT; @@ -62,10 +63,10 @@ static void usage(void) { - fprintf(stderr, "usage: %s create [-v] [-o ] " + fprintf(stderr, "usage: %s create [-v] [-o ] [-r ] " "[-s sectorsize] [-t timeout] [-u unit] \n", getprogname()); - fprintf(stderr, " %s rescue [-v] [-o ] <-u unit> " - "\n", getprogname()); + fprintf(stderr, " %s rescue [-v] [-o ] [-r ] " + "<-u unit> \n", getprogname()); fprintf(stderr, " %s destroy [-f] <-u unit>\n", getprogname()); fprintf(stderr, " %s list [-v] [-u unit]\n", getprogname()); exit(EXIT_FAILURE); @@ -87,6 +88,7 @@ g_gatel_serve(int fd) { struct g_gate_ctl_io ggio; size_t bsize; + char ident[DISK_IDENT_SIZE]; if (g_gate_verbose == 0) { if (daemon(0, 0) == -1) { @@ -99,8 +101,15 @@ g_gatel_serve(int fd) ggio.gctl_unit = unit; bsize = sectorsize; ggio.gctl_data = malloc(bsize); + + errno = 0; + if (retries && ioctl(fd, DIOCGIDENT, ident) != 0) { + g_gate_xlog("Failed to get disk ident for %s: %s", path, + strerror(errno)); + } for (;;) { int error; + int retries_left; once_again: ggio.gctl_length = bsize; ggio.gctl_error = 0; @@ -132,6 +141,8 @@ g_gatel_serve(int fd) strerror(error)); } + retries_left = retries; +retry_request: error = 0; switch (ggio.gctl_cmd) { case BIO_READ: @@ -147,6 +158,11 @@ g_gatel_serve(int fd) if (pread(fd, ggio.gctl_data, ggio.gctl_length, ggio.gctl_offset) == -1) { error = errno; + g_gate_log(LOG_ERR, "Failed to read %d" + " bytes at offset %d from %s: %s", + ggio.gctl_length, + (intmax_t)ggio.gctl_offset, path, + strerror(error)); } } break; @@ -155,12 +171,47 @@ g_gatel_serve(int fd) if (pwrite(fd, ggio.gctl_data, ggio.gctl_length, ggio.gctl_offset) == -1) { error = errno; + g_gate_log(LOG_ERR, "Failed to write %d bytes" + " at offset %jd to %s: %s", + ggio.gctl_length, + (intmax_t)ggio.gctl_offset, path, + strerror(error)); } break; default: error = EOPNOTSUPP; } + if (error && error != EOPNOTSUPP) { + if (retries_left > 0) { + char ident_new[DISK_IDENT_SIZE]; + + close(fd); + retries_left--; + sleep(5); + fd = open(path, g_gate_openflags(flags) | + O_DIRECT | O_FSYNC); + if (fd == -1) { + err(EXIT_FAILURE, "Cannot open %s", + path); + } + if (ioctl(fd, DIOCGIDENT, ident_new) != 0) { + g_gate_xlog("Failed to get disk ", + "ident for %s: %s", path, + strerror(errno)); + } + if (strcmp(ident, ident_new) != 0) { + g_gate_xlog("Disk ident for %s " + "changed from %s to %s. Reuse " + "could cause data loss.", path, + ident, ident_new); + } + g_gate_log(LOG_ERR, "Retrying after reopening " + "%s (%s). Retries left: %d", path, ident, + retries_left); + goto retry_request; + } + } ggio.gctl_error = error; g_gate_ioctl(G_GATE_CMD_DONE, &ggio); } @@ -232,7 +283,7 @@ main(int argc, char *argv[]) for (;;) { int ch; - ch = getopt(argc, argv, "fo:s:t:u:v"); + ch = getopt(argc, argv, "fo:r:s:t:u:v"); if (ch == -1) break; switch (ch) { @@ -255,6 +306,14 @@ main(int argc, char *argv[]) "Invalid argument for '-o' option."); } break; + case 'r': + if (action != CREATE && action != RESCUE) + usage(); + errno = 0; + retries = strtoul(optarg, NULL, 10); + if (retries == 0 && errno != 0) + errx(EXIT_FAILURE, "Invalid retry count."); + break; case 's': if (action != CREATE) usage(); -- 2.32.0 From a2af2cc28004bb72a3665d6cbc1a2cc11a8c929f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 26 Apr 2015 17:54:14 +0200 Subject: [PATCH 153/325] ggatel.8: Document the shiny new -r option Obtained from: ElectroBSD --- sbin/ggate/ggatel/ggatel.8 | 61 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/sbin/ggate/ggatel/ggatel.8 b/sbin/ggate/ggatel/ggatel.8 index 7facee0c5369..d80a27eb54a4 100644 --- a/sbin/ggate/ggatel/ggatel.8 +++ b/sbin/ggate/ggatel/ggatel.8 @@ -35,6 +35,7 @@ .Cm create .Op Fl v .Op Fl o Cm ro | wo | rw +.Op Fl r Ar retries .Op Fl s Ar sectorsize .Op Fl t Ar timeout .Op Fl u Ar unit @@ -51,6 +52,7 @@ .Cm rescue .Op Fl v .Op Fl o Cm ro | wo | rw +.Op Fl r Ar retries .Fl u Ar unit .Ar path .Sh DESCRIPTION @@ -103,6 +105,19 @@ or read-write .Pq Cm rw . Default is .Cm rw . +.It Fl r Ar retries +Number of times a failed request should be retried before forwarding +the error to the kernel. +Between retries, +.Nm ggatel +waits for five seconds and reopens the device in case it temporarily +disappeared. +The reopened device is only used if the disk identification did not +change. +This option is useful when using unreliable USB devices as geli +consumer (as long as the device loss does not cause the USB +stack to deadlock). +By default failed requests are not retried. .It Fl s Ar sectorsize Sector size for .Nm ggate @@ -145,6 +160,52 @@ umount /secret gbde detach ggate5 ggatel destroy -u 5 .Ed + +Scrub a pool on an USB device that occasionally disappears: +.Bd -literal -offset indent +$ glabel list da0 +Geom name: da0 +Providers: +1. Name: label/extreme + Mediasize: 4023385600 (3.7G) + Sectorsize: 512 + Mode: r0w0e0 + secoffset: 0 + offset: 0 + seclength: 7858175 + length: 4023385600 + index: 0 +Consumers: +1. Name: da0 + Mediasize: 4023386112 (3.7G) + Sectorsize: 512 + Mode: r0w0e0 + +$ sudo ggatel create -r 2 /dev/da0 +ggate0 +$ glabel list da0 +glabel: No such geom: da0. +glabel list ggate0 +Geom name: ggate0 +Providers: +1. Name: label/extreme + Mediasize: 4023385600 (3.7G) + Sectorsize: 512 + Mode: r0w0e0 + secoffset: 0 + offset: 0 + seclength: 7858175 + length: 4023385600 + index: 0 +Consumers: +1. Name: ggate0 + Mediasize: 4023386112 (3.7G) + Sectorsize: 512 + Mode: r0w0e0 + +$ zogftw import extreme +$ sudo zpool scrub extreme +.Ed .Sh SEE ALSO .Xr geom 4 , .Xr gbde 8 , -- 2.32.0 From 7df3a3a1998195fcfe83d2ea1db8cf4509503347 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 26 Mar 2017 18:42:38 +0200 Subject: [PATCH 154/325] ELECTRO_BLOAT: Disable GEOM_RAID which can cause deadlocks ... even if it's not intentionally being used: 13 100020 geom g_event mi_switch+0xd2 sleepq_wait+0x3a _sleep+0x257 biowait+0x90 g_read_data+0x7e g_raid_md_taste_sii+0xde g_raid_taste+0x16b g_new_provider_event+0xca g_run_events+0x186 fork_exit+0x85 fork_trampoline+0xe 31379 100863 zpool - mi_switch+0xd2 sleepq_timedwait+0x42 _sleep+0x237 g_waitidle+0xa1 userret+0x55 amd64_syscall+0x52f Xfast_syscall+0xfb 60137 101433 ggatel - mi_switch+0xd2 sleepq_timedwait+0x42 _sleep+0x237 g_waitidle+0xa1 userret+0x55 amd64_syscall+0x52f Xfast_syscall+0xfb Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index 3bd60d2df610..f440d4222d02 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -45,7 +45,11 @@ options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. -options GEOM_RAID # Soft RAID functionality. +# This is unlikely to be intentionally used be ElectroBSD +# users but its mere presence in the kernel can result in +# deadlocks while tasting flaky providers and the deadlock +# affects other geom classes as well. +nooptions GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization options COMPAT_FREEBSD32 # Compatible with i386 binaries options COMPAT_FREEBSD10 # Compatible with FreeBSD10 -- 2.32.0 From 70826d87135e1062e1432428a0a3a1e905bce9b4 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 1 Apr 2017 17:35:26 +0200 Subject: [PATCH 155/325] release/scripts/image-checksum.sh: Add a bunch of additional mtree keywords to see if they make a difference Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 30ac62a6c168..18e3a521880f 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -49,7 +49,7 @@ UFS_PARTITION=p2 EXPECTED_PARTITIONS=2 MOUNTPOINT=/mnt VERBOSE=0 -MTREE_KEYWORDS=size,time,uid,gid,sha256 +MTREE_KEYWORDS=size,time,mode,uid,gid,sha256,device,flags,link,nlink,type REUSE_EXISTING_CACHE_FILES=false verbose_log() { -- 2.32.0 From 85e3fa335bf0e5acf47fe6bb08c7b2418b61ab36 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 7 Apr 2017 09:33:34 +0200 Subject: [PATCH 156/325] Let image-checksum check inodes using ls Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 32 +++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index 18e3a521880f..b8c1c94d0aaa 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -76,6 +76,22 @@ create_mtree_spec_file() { umount "${MOUNTPOINT}" || return 1 } +create_ls_file() { + local md_unit ls_file + + md_unit="${1}" + ls_file="${2}" + + verbose_log "Mounting /dev/md${md_unit}${UFS_PARTITION} at ${MOUNTPOINT}" + mount -o ro "/dev/md${md_unit}${UFS_PARTITION}" "${MOUNTPOINT}" || return 1 + + verbose_log "Running ls to get inodes, saving output in ${ls_file}" + env -i LC_ALL=C ls -liR "${MOUNTPOINT}" > "${ls_file}" || return 1 + + verbose_log "Unmounting ${MOUNTPOINT} ..." + umount "${MOUNTPOINT}" || return 1 +} + partition_count_acceptable() { local md_unit="${1}" @@ -89,7 +105,9 @@ partition_count_acceptable() { generate_partial_image_checksum() { local image_file \ - md_unit spec_file gpart_file + md_unit spec_file gpart_file ls_file \ + gpart_checksum bootcode_checksum ls_checksum \ + weak_image_checksum mtree_checksum image_file="${1}" spec_file="${image_file}.mtree" @@ -102,6 +120,11 @@ generate_partial_image_checksum() { echo "gpart file ${gpart_file} already exists" ${REUSE_EXISTING_CACHE_FILES} || return 1 fi + ls_file="${image_file}.ls" + if [ -f "${ls_file}" ]; then + echo "ls file ${ls_file} already exists" + ${REUSE_EXISTING_CACHE_FILES} || return 1 + fi md_unit=$(mdconfig -o readonly -n -f "${image_file}") if [ $? != 0 ]; then @@ -116,6 +139,9 @@ generate_partial_image_checksum() { if [ ! -f "${gpart_file}" ]; then gpart list "md${md_unit}" | sed -E -e "s@(: md)${md_unit}@\1X@" > "${gpart_file}" fi + if [ ! -f "${ls_file}" ]; then + create_ls_file "${md_unit}" "${ls_file}" || return 1 + fi gpart_checksum=$(sha256 -q "${gpart_file}") verbose_log "gpart checksum: ${gpart_checksum}" @@ -126,8 +152,10 @@ generate_partial_image_checksum() { verbose_log "Boot code checksum: ${bootcode_checksum}" mtree_checksum=$(sha256 -q "${spec_file}") verbose_log "mtree checksum: ${mtree_checksum}" + ls_checksum=$(sha256 -q "${ls_file}") + verbose_log "ls checksum: ${ls_checksum}" - weak_image_checksum=$(echo "${gpart_checksum} ${bootcode_checksum} ${mtree_checksum}" | sha256) + weak_image_checksum=$(echo "${gpart_checksum} ${bootcode_checksum} ${mtree_checksum} ${ls_checksum}" | sha256) echo "Partial image checksum for ${image_file}: ${weak_image_checksum}" } -- 2.32.0 From 663a125380234f04dae4e656cfb6eb708caace62 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 4 Apr 2017 14:29:31 +0200 Subject: [PATCH 157/325] release/scripts/strip-freebsd.sh: Ditch qlnx ... which contains source-less microcode in sys/dev/qlnx/qlnxe/ecore_init_values.h. It was added in r316485/c52917fd11 but not MFC'd yet. This commit should make sure that the MFC will break the ElectroBSD built and can be fixed if the upstream report is ignored. Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index e805f9cab8e4..c6875e6baab8 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -86,7 +86,7 @@ get_tainted_sys_contrib_devs() { # and may cause build failures without it. get_tainted_sys_devs() { # bce has already been taken care of by get_bce_files() above - echo "bxe ctau cx cxgb cxgbe ispfw qlxgbe" \ + echo "bxe ctau cx cxgb cxgbe ispfw qlxgbe qlnx" \ "it tw" } -- 2.32.0 From 5525cc9d59df5b732c33650713b25e0d2502aa70 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 7 Apr 2017 15:46:05 +0200 Subject: [PATCH 158/325] release/Makefile: Try to debug /etc/rc.local mode issue and fix it Obtained from: ElectroBSD --- release/Makefile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/release/Makefile b/release/Makefile index 160708bef2cf..89dcccf86285 100644 --- a/release/Makefile +++ b/release/Makefile @@ -213,6 +213,10 @@ disc1: packagesystem echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf cp ${.CURDIR}/rc.local ${.TARGET}/etc + umask + ls -l ${.TARGET}/etc/rc.local + chmod 550 ${.TARGET}/etc/rc.local + ls -l ${.TARGET}/etc/rc.local touch ${.TARGET} bootonly: packagesystem -- 2.32.0 From 863e043adcc5c4b94272a8169428441838dca11a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 8 Apr 2017 15:41:03 +0200 Subject: [PATCH 159/325] usr.sbin/cdcontrol: Retry closing after ENOTTY failures (XXX: not really) This doesn't actually work but the commit is kept around as a reminder to fix it. Obtained from: ElectroBSD --- usr.sbin/cdcontrol/cdcontrol.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/usr.sbin/cdcontrol/cdcontrol.c b/usr.sbin/cdcontrol/cdcontrol.c index 546c20b6f94a..0fa1f7a02534 100644 --- a/usr.sbin/cdcontrol/cdcontrol.c +++ b/usr.sbin/cdcontrol/cdcontrol.c @@ -389,6 +389,10 @@ run(int cmd, char *arg) (void) ioctl (fd, CDIOCALLOW); rc = ioctl (fd, CDIOCCLOSE); + if (rc == ENOTTY) { + warnx("Retrying"); + rc = ioctl (fd, CDIOCCLOSE); + } if (rc < 0) return (rc); close(fd); -- 2.32.0 From d9816cf84ffb21ba549862bd3804ebff38ef11c6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 16 Apr 2017 13:35:11 +0200 Subject: [PATCH 160/325] Add ELECTRO_BEER_DEBUG kernel which is ELECTRO_BEER plus debug options Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BEER_DEBUG | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BEER_DEBUG diff --git a/sys/amd64/conf/ELECTRO_BEER_DEBUG b/sys/amd64/conf/ELECTRO_BEER_DEBUG new file mode 100644 index 000000000000..22c1b306669a --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BEER_DEBUG @@ -0,0 +1,8 @@ +include ELECTRO_BEER + +ident ELECTRO_BEER_DEBUG + +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed -- 2.32.0 From 6186077401d4b3d756d5f787ad94087d1cf0a98f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 20 Aug 2015 10:12:27 +0200 Subject: [PATCH 161/325] pw(8): Increase minimal random password length to 32 It's not obvious to me why the length is randomized as well in the first place but whatever ... Obtained from: ElectroBSD --- usr.sbin/pw/pw_user.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/usr.sbin/pw/pw_user.c b/usr.sbin/pw/pw_user.c index 2eec317b5e5b..552ce78a6665 100644 --- a/usr.sbin/pw/pw_user.c +++ b/usr.sbin/pw/pw_user.c @@ -514,13 +514,13 @@ static char * pw_password(struct userconf * cnf, char const * user, bool dryrun) { int i, l; - char pwbuf[32]; + char pwbuf[41]; switch (cnf->default_password) { case P_NONE: /* No password at all! */ return ""; case P_RANDOM: /* Random password */ - l = (arc4random() % 8 + 8); /* 8 - 16 chars */ + l = sizeof(pwbuf) - 1 - (arc4random() % 8); /* 32 - 40 chars */ for (i = 0; i < l; i++) pwbuf[i] = chars[arc4random_uniform(sizeof(chars)-1)]; pwbuf[i] = '\0'; -- 2.32.0 From 7ae1d9b7e669201fe1e1720fdfcbdc6ad48910b3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 14 May 2016 16:53:50 +0200 Subject: [PATCH 162/325] uma.h: Rename uma_prealloc()'s second argument to nitems to increase consistency PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209716 PR submission date: 2016-05-23 Obtained from: ElectroBSD --- sys/vm/uma.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/vm/uma.h b/sys/vm/uma.h index 620bba48a8eb..9b1af20bdf2d 100644 --- a/sys/vm/uma.h +++ b/sys/vm/uma.h @@ -627,14 +627,14 @@ void uma_zone_set_freef(uma_zone_t zone, uma_free freef); * * Arguments: * zone The zone to fill - * itemcnt The number of items to reserve + * nitems The number of items to reserve * * Returns: * Nothing * * NOTE: This is blocking and should only be done at startup */ -void uma_prealloc(uma_zone_t zone, int itemcnt); +void uma_prealloc(uma_zone_t zone, int nitems); /* * Used to determine if a fixed-size zone is exhausted. -- 2.32.0 From 6cd354babd4813a423a44f737bf4a30a36058b2e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 16 May 2016 12:28:50 +0200 Subject: [PATCH 163/325] uma.h: Stop claiming that uma_zcreate() may return NULL ... 'if the wait flag is not set'. The function does not actually accept a "wait flag". Internally it unconditionally uses the M_WAITOK flag when calling zone_alloc_item(). Obtained from: ElectroBSD --- sys/vm/uma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/vm/uma.h b/sys/vm/uma.h index 9b1af20bdf2d..335ee48b7239 100644 --- a/sys/vm/uma.h +++ b/sys/vm/uma.h @@ -176,7 +176,7 @@ typedef void (*uma_release)(void *arg, void **store, int count); * * Returns: * A pointer to a structure which is intended to be opaque to users of - * the interface. The value may be null if the wait flag is not set. + * the interface. */ uma_zone_t uma_zcreate(const char *name, size_t size, uma_ctor ctor, uma_dtor dtor, uma_init uminit, uma_fini fini, -- 2.32.0 From 0a6c666cf9f124a2fd9016c8a971b0e24edc9972 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 24 Apr 2017 21:45:28 +0200 Subject: [PATCH 164/325] Add ELECTRO_BLOAT_DEBUG which is ELECTRO_BLOAT plus debug settings Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT_DEBUG | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BLOAT_DEBUG diff --git a/sys/amd64/conf/ELECTRO_BLOAT_DEBUG b/sys/amd64/conf/ELECTRO_BLOAT_DEBUG new file mode 100644 index 000000000000..2d06dcf4e2d8 --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BLOAT_DEBUG @@ -0,0 +1,9 @@ +include ELECTRO_BLOAT + +ident ELECTRO_BLOAT_DEBUG + +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +options DEBUG_MEMGUARD -- 2.32.0 From ede65e6dbd1b23a840e3772af05f97c530162db8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 17 Mar 2016 12:51:02 +0100 Subject: [PATCH 165/325] sys/cam: Retry in case of 'uncorrectable' errors At least in case of the LITE-ON DVDRW SOHW-1693S with firmware KC4B, they are recoverable. Maybe this should be done as device-specific quirk. Obtained from: ElectroBSD --- sys/cam/scsi/scsi_all.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/cam/scsi/scsi_all.c b/sys/cam/scsi/scsi_all.c index 6c8dec7cc235..49644532283a 100644 --- a/sys/cam/scsi/scsi_all.c +++ b/sys/cam/scsi/scsi_all.c @@ -1412,8 +1412,8 @@ static struct asc_table_entry asc_table[] = { { SST(0x11, 0x04, SS_FATAL|EIO, "Unrecovered read error - auto reallocate failed") }, /* WRO B */ - { SST(0x11, 0x05, SS_FATAL|EIO, - "L-EC uncorrectable error") }, + { SST(0x11, 0x05, SS_RDEF, + "L-EC 'uncorrectable' error") }, /* WRO B */ { SST(0x11, 0x06, SS_FATAL|EIO, "CIRC unrecovered error") }, -- 2.32.0 From ad2b62e932b3d69bb75835f28e383f1d1be8cb23 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 27 Apr 2017 19:45:55 +0200 Subject: [PATCH 166/325] kern_racct: Add sysctl to modify the delay between resource limit checks etc. Obtained from: ElectroBSD --- sys/kern/kern_racct.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/sys/kern/kern_racct.c b/sys/kern/kern_racct.c index d1ec1db880d4..8d67788ad75e 100644 --- a/sys/kern/kern_racct.c +++ b/sys/kern/kern_racct.c @@ -85,6 +85,11 @@ SYSCTL_UINT(_kern_racct, OID_AUTO, enable, CTLFLAG_RDTUN, &racct_enable, SYSCTL_UINT(_kern_racct, OID_AUTO, pcpu_threshold, CTLFLAG_RW, &pcpu_threshold, 0, "Processes with higher %cpu usage than this value can be throttled."); +static unsigned int racctd_delay = 0; +SYSCTL_UINT(_kern_racct, OID_AUTO, racctd_delay, CTLFLAG_RWTUN, + &racctd_delay, 0, "Number of ticks to pause between resource " + "usage/limit updates."); + /* * How many seconds it takes to use the scheduler %cpu calculations. When a * process starts, we compute its %cpu usage by dividing its runtime by the @@ -1223,6 +1228,9 @@ racctd(void) ASSERT_RACCT_ENABLED(); + if (racctd_delay == 0) + racctd_delay = hz; + for (;;) { racct_decay(); @@ -1302,7 +1310,7 @@ racctd(void) PROC_UNLOCK(p); } sx_sunlock(&allproc_lock); - pause("-", hz); + pause("-", racctd_delay); } } -- 2.32.0 From 567f5bf8a8d43e28563c0c88bc0fb22178a03d30 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 19 May 2017 11:53:57 +0200 Subject: [PATCH 167/325] sys/net: Disable MAC address caching by default MAC address caching was added upstream in r318160/0cfdb3c3056d9e and allows to retrieve the orginal MAC address with ifconfig, even if it has been previously overwritten. This currently can be done from jails and makes system fingerprinting easier which may be undesirable. Now it has to be explicitly enabled by setting: net.link.cache_mac_addresses=1 before the NIC is initialized. Note that jails can still access the (potentially randomized) MAC address the NICs are currently using. The "protection" offered by this commit is therefore quite limitted. Obtained from: ElectroBSD --- sys/net/if.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/sys/net/if.c b/sys/net/if.c index f2ef88d3f28e..889fccc1a011 100644 --- a/sys/net/if.c +++ b/sys/net/if.c @@ -220,6 +220,13 @@ SYSCTL_UINT(_net, OID_AUTO, ifdescr_maxlen, CTLFLAG_RW, &ifdescr_maxlen, 0, "administrative maximum length for interface description"); +/* Try to cache the original MAC addresses */ +static int cache_mac_addresses = 0; +SYSCTL_INT(_net_link, OID_AUTO, cache_mac_addresses, CTLFLAG_RWTUN, + &cache_mac_addresses, 0, + "Try to cache original MAC addresses allowing " + "ifconfig to display them after being overwritten."); + static MALLOC_DEFINE(M_IFDESCR, "ifdescr", "ifnet descriptions"); /* global sx for non-critical path ifdescr */ @@ -917,7 +924,7 @@ if_attach_internal(struct ifnet *ifp, int vmove, struct if_clone *ifc) /* Reliably crash if used uninitialized. */ ifp->if_broadcastaddr = NULL; - if (ifp->if_type == IFT_ETHER) { + if (cache_mac_addresses != 0 && ifp->if_type == IFT_ETHER) { ifp->if_hw_addr = malloc(ifp->if_addrlen, M_IFADDR, M_WAITOK | M_ZERO); } -- 2.32.0 From f30a3910ae365bea0a7bd3cc5c79b7c0197e1e8b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 21 May 2017 14:27:06 +0200 Subject: [PATCH 168/325] cddl/lib/libdtrace: Remove -O2 for now to get better core dumps Obtained from: ElectroBSD --- cddl/lib/libdtrace/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/cddl/lib/libdtrace/Makefile b/cddl/lib/libdtrace/Makefile index bc58855ffc5d..cab96044b23e 100644 --- a/cddl/lib/libdtrace/Makefile +++ b/cddl/lib/libdtrace/Makefile @@ -74,6 +74,7 @@ CFLAGS+= -I${.OBJDIR} -I${.CURDIR} \ -I${OPENSOLARIS_USR_DISTDIR}/lib/libctf/common \ -I${OPENSOLARIS_USR_DISTDIR}/lib/libdtrace/common \ -I${OPENSOLARIS_SYS_DISTDIR}/uts/common +CFLAGS:= ${CFLAGS:S/-O2//} #CFLAGS+= -DYYDEBUG -- 2.32.0 From 1c0904da575df3888b88a5d0ac32bb4ba3886b77 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 1 Jun 2017 16:45:03 +0200 Subject: [PATCH 169/325] Don't try to build cxgbetool It's not useful without cxgbe which isn't shipped with ElectroBSD and relies on t4_ioctl.h which isn't shipped either. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index a59aefe66d3b..0117d6594697 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -392,14 +392,9 @@ BROKEN_OPTIONS+=LOADER_GELI LOADER_LUA # profiling won't work on MIPS64 because there is only assembly for o32 BROKEN_OPTIONS+=PROFILE .endif -.if ${__T} == "aarch64" || ${__T} == "amd64" || ${__T} == "i386" || \ - ${__T} == "powerpc64" || ${__T} == "sparc64" -__DEFAULT_YES_OPTIONS+=CXGBETOOL -__DEFAULT_YES_OPTIONS+=MLX5TOOL -.else + __DEFAULT_NO_OPTIONS+=CXGBETOOL __DEFAULT_NO_OPTIONS+=MLX5TOOL -.endif # HyperV is currently x86-only .if ${__T} == "amd64" || ${__T} == "i386" -- 2.32.0 From 3bdc00094503b3e42d3177f5fd02e2031a6bcade Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 24 Jun 2017 12:04:15 +0200 Subject: [PATCH 170/325] Detach upgt from the build It relies on proprietary firmware (not part of the upstream tree). Obtained from: ElectroBSD --- sys/modules/usb/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/modules/usb/Makefile b/sys/modules/usb/Makefile index 239463f95125..b515fcf811a9 100644 --- a/sys/modules/usb/Makefile +++ b/sys/modules/usb/Makefile @@ -46,7 +46,7 @@ MAKE+=" DEBUG_FLAGS+=-DUSB_REQ_DEBUG" SUBDIR = usb SUBDIR += ${_dwc_otg} ehci ${_musb} ohci uhci xhci ${_uss820dci} \ ${_atmegadci} ${_avr32dci} ${_rsu} ${_rsufw} ${_saf1761otg} -SUBDIR += ${_rum} ${_run} ${_runfw} ${_uath} upgt usie ural ${_zyd} ${_urtw} +SUBDIR += ${_rum} ${_run} ${_runfw} ${_uath} usie ural ${_zyd} ${_urtw} SUBDIR += atp cfumass uhid uhid_snes ukbd ums udbp ufm uep wmt wsp ugold uled SUBDIR += ucom u3g uark ubsa ubser uchcom ucycom ufoma uftdi ugensa uipaq ulpt \ umct umcs umodem umoscom uplcom uslcom uvisor uvscom -- 2.32.0 From 7fdfe7c50b75ba7928d30075194bdb234bc3ac5c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 10 Jul 2017 14:37:11 +0200 Subject: [PATCH 171/325] sys/kern: Follow OpenBSD's lead and remove TIOCSTI support ... to prevent tty hijacking issues like CVE-2005-4890. TIOCSTI is still used by by mail, but this could (probably) be fixed by adding a "#undef TIOCSTI" or removing the definition from sys/sys/ttycom.h. Additionally TIOCSTI is still used in tcsh, but as tcsh isn't compiled on ElectroBSD we don't care. Obtained from: ElectroBSD --- sys/kern/tty.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/sys/kern/tty.c b/sys/kern/tty.c index f80ffd4921e6..a1f303865318 100644 --- a/sys/kern/tty.c +++ b/sys/kern/tty.c @@ -576,7 +576,6 @@ ttydev_ioctl(struct cdev *dev, u_long cmd, caddr_t data, int fflag, case TIOCSPGRP: case TIOCSTART: case TIOCSTAT: - case TIOCSTI: case TIOCSTOP: case TIOCSWINSZ: #if 0 @@ -1951,14 +1950,7 @@ tty_generic_ioctl(struct tty *tp, u_long cmd, void *data, int fflag, tty_info(tp); return (0); case TIOCSTI: - if ((fflag & FREAD) == 0 && priv_check(td, PRIV_TTY_STI)) - return (EPERM); - if (!tty_is_ctty(tp, td->td_proc) && - priv_check(td, PRIV_TTY_STI)) - return (EACCES); - ttydisc_rint(tp, *(char *)data, 0); - ttydisc_rint_done(tp); - return (0); + return (EIO); } #ifdef COMPAT_43TTY -- 2.32.0 From b459b8fa1e7d165edd5c24266cacdbe6e32c43d6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 28 Jul 2017 12:07:47 +0200 Subject: [PATCH 172/325] cardbus: Do not give the world read and write permission to the cardbus device Obtained from: ElectroBSD --- sys/dev/cardbus/cardbus_device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/cardbus/cardbus_device.c b/sys/dev/cardbus/cardbus_device.c index de0a53a93f21..88ffbe4f5998 100644 --- a/sys/dev/cardbus/cardbus_device.c +++ b/sys/dev/cardbus/cardbus_device.c @@ -117,7 +117,7 @@ cardbus_device_create(struct cardbus_softc *sc, struct cardbus_devinfo *devi, cardbus_device_buffer_cis(parent, child, &devi->sc_cis); minor = (device_get_unit(sc->sc_dev) << 8) + devi->pci.cfg.func; unit = device_get_unit(sc->sc_dev); - devi->sc_cisdev = make_dev(&cardbus_cdevsw, minor, 0, 0, 0666, + devi->sc_cisdev = make_dev(&cardbus_cdevsw, minor, 0, 0, 0660, "cardbus%d.%d.cis", unit, devi->pci.cfg.func); if (devi->pci.cfg.func == 0) make_dev_alias(devi->sc_cisdev, "cardbus%d.cis", unit); -- 2.32.0 From ec9498c691b94b9366bea1e94e282a4360f05a53 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 4 Aug 2017 12:06:37 +0200 Subject: [PATCH 173/325] kern_fcntl(): Unconditionally init tmp Obtained from: ElectroBSD --- sys/kern/kern_descrip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/kern_descrip.c b/sys/kern/kern_descrip.c index 6e97a1e8e2f5..5dc65d2d7723 100644 --- a/sys/kern/kern_descrip.c +++ b/sys/kern/kern_descrip.c @@ -494,7 +494,7 @@ kern_fcntl(struct thread *td, int fd, int cmd, intptr_t arg) uint64_t bsize; off_t foffset; - error = 0; + tmp = error = 0; flg = F_POSIX; p = td->td_proc; fdp = p->p_fd; -- 2.32.0 From 2ea204fda918ad901e626babba2f04ade8bbb6e3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 6 Aug 2017 13:32:21 +0200 Subject: [PATCH 174/325] sys/kern/sched_ule.c: Don't add an empty line to the generated spec Obtained from: ElectroBSD --- sys/kern/sched_ule.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c index ed5cef8ab80a..af367fdb4a3f 100644 --- a/sys/kern/sched_ule.c +++ b/sys/kern/sched_ule.c @@ -3031,7 +3031,7 @@ sysctl_kern_sched_topology_spec(SYSCTL_HANDLER_ARGS) sbuf_printf(topo, "\n"); err = sysctl_kern_sched_topology_spec_internal(topo, cpu_top, 1); - sbuf_printf(topo, "\n"); + sbuf_printf(topo, ""); if (err == 0) { err = sbuf_finish(topo); -- 2.32.0 From 07d572f2795719cec48effb8aff1776698f1b38c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 18 Jul 2017 21:33:50 +0200 Subject: [PATCH 175/325] sys/amd64/conf/ELECTRO_BLOAT: Add options CAM_IOSCHED_DYNAMIC to see what happens Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index f440d4222d02..8016f8898070 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -16,6 +16,9 @@ options ACCEPT_FILTER_HTTP # works around various bugs that only affect the module build. device pf +# XXX: Not yet proven to be useful +options CAM_IOSCHED_DYNAMIC + ############################################################################## # Everything below comes from GENERIC, but "offending" lines have been removed ############################################################################## -- 2.32.0 From c56768a4a092e892232bea58e9fc480223e780ea Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 29 May 2015 10:46:06 +0200 Subject: [PATCH 176/325] Import cloudiatr 2021-11-08-ca178a4 Obtained from: ElectroBSD --- usr.sbin/Makefile | 1 + usr.sbin/cloudiatr/Makefile | 3 + usr.sbin/cloudiatr/cloudiatr | 1411 ++++++++++++++++++++++++++++++++++ 3 files changed, 1415 insertions(+) create mode 100644 usr.sbin/cloudiatr/Makefile create mode 100755 usr.sbin/cloudiatr/cloudiatr diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index 33e81b4f64ac..527002a6b04b 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -7,6 +7,7 @@ SUBDIR= adduser \ arp \ binmiscctl \ camdd \ + cloudiatr \ cdcontrol \ chkgrp \ chown \ diff --git a/usr.sbin/cloudiatr/Makefile b/usr.sbin/cloudiatr/Makefile new file mode 100644 index 000000000000..34ada8582ecd --- /dev/null +++ b/usr.sbin/cloudiatr/Makefile @@ -0,0 +1,3 @@ +SCRIPTS= cloudiatr + +.include diff --git a/usr.sbin/cloudiatr/cloudiatr b/usr.sbin/cloudiatr/cloudiatr new file mode 100755 index 000000000000..87de7dfbc5c1 --- /dev/null +++ b/usr.sbin/cloudiatr/cloudiatr @@ -0,0 +1,1411 @@ +#!/bin/sh + +########################################################################### +# cloudiatr +# +# Buzzword-compliant remote OS eviction tool. For details see: +# https://www.fabiankeil.de/gehacktes/cloudiatr/ +# +# Copyright (c) 2014-2021 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ALL YOUR +# DATA IS BELONG TO THE SOFTWARE AND MAY BE EATEN BY IT. IF THAT IS NOT +# ACCEPTABLE, YOU SHOULD PROBABLY MAKE BACKUPS BEFORE USING THE SOFTWARE. +########################################################################### + +# These subcommands skip the privilege check. +# +# The "cmd" subcommand may actually required privileges +# depending on the arguments but the check isn't strictly +# required and skipping it for all arguments is more convenient +# than enforcing it for all. +# +# The variable can't be set through cloudiatr_init_globals +# as it's needed earlier. +CLOUDIATR_UNPRIVILEGED_SUBCOMMANDS="cmd show-config version" + +# It's important that this function is called before any other +# function except cloudiatr_main(), otherwise fatal errors may +# not be caught. +cloudiatr_init() { + local mode="${1}" + + set -e + cloudiatr_init_globals + + cloudiatr_load_config_file "${CLOUDIATR_CONFIG_FILE}" + + cloudiatr_check_config +} + +cloudiatr_fyi() { + local message="${*}" + + echo "cloudiatr: $message" +} + +cloudiatr_wtf() { + local complaints="${*}" + if [ -z "${complaints}" ]; then + complaints="cloudiatr_wtf(): No complaints?" + fi + cloudiatr_fyi "${complaints}" 1>&2 + return 1 +} + +cloudiatr_check_config() { + local v_flag \ + mandatory_variable optional_variable value fail + + v_flag="${1}" + fail=0 + + for mandatory_variable in ${CLOUDIATR_MANDATORY_VARIABLES}; do + value="$(eval 'echo $'"${mandatory_variable}")" + if [ -z "${value}" ]; then + cloudiatr_wtf "Fatal error: ${mandatory_variable} is unset" + fail=1 + elif [ "${v_flag}" = "-v" ]; then + echo "${mandatory_variable}='${value}'" + fi + done + if [ "${v_flag}" = "-v" ]; then + for optional_variable in ${CLOUDIATR_OPTIONAL_VARIABLES}; do + value="$(eval 'echo $'"${optional_variable}")" + echo "${optional_variable}='${value}'" + done + fi + return $fail +} + +cloudiatr_show_config() { + cloudiatr_check_config -v +} + +cloudiatr_load_config_file() { + local config_file="${1}" + + if [ -f "${config_file}" ]; then + . "${config_file}" + return 0 + fi + cloudiatr_wtf "Config file ${config_file} does not exist. You can use '$0 -f path/to/file ...' to specify a different one" + return 1 +} + +cloudiatr_init_globals() { + + CLOUDIATR_VERSION="2021-11-08-ca178a4" + + CLOUDIATR_NEW_SYSTEM_DIR=/cloudiatr + # Only needs to be enough for a stripped-down bootfs + CLOUDIATR_BPOOL_PARTITION_SIZE=200M + # Has to be enough for the rest of the OS including the "permanent" + # /boot that is only used to (re)populate the bootfs on the bpool. + CLOUDIATR_RPOOL_PARTITION_SIZE=4G + CLOUDIATR_SWAP_PARTITION_SIZE=4G + + # Flags passed to zpool create when creating the boot and root pools. + # Creating the pools with version 28 makes recovery with outdated + # recovery systems more convenient. + CLOUDIATR_BPOOL_CREATE_FLAGS="-o version=28 -O compression=on -f" + CLOUDIATR_RPOOL_CREATE_FLAGS="-o version=28 -O compression=on -O checksum=sha256" + + # Set to true to use the existing partition layout. + # + # Only expected to work if the layout was created by a previous + # cloudiatr run. By default the partitions 2, 3 and 4 will be + # overwritten and partition 1 is expected to contain working bootcode. + CLOUDIATR_REUSE_GPART_SETUP=false + + # Usually changing the partition numbers usually is not necessary + # and changing them after the installation is likely to result in + # data loss. + CLOUDIATR_BOOTCODE_PARTITION=1 + CLOUDIATR_BPOOL_PARTITION=2 + CLOUDIATR_RPOOL_PARTITION=3 + CLOUDIATR_SWAP_PARTITION=4 + CLOUDIATR_DPOOL_PARTITION=5 + + CLOUDIATR_BPOOL_NAME="bpool" + CLOUDIATR_RPOOL_NAME="rpool" + + # Note that DEFAULT is a fallback documented in rc.conf(5). + # Not changing this variable to the actual network interface + # is likely to cause problems if there are more than one nics + # and you aren't using DHCP for all of them. + CLOUDIATR_NIC="DEFAULT" + + # Will be created + CLOUDIATR_RPOOL_KEY_NAME="${CLOUDIATR_RPOOL_NAME}.key" + CLOUDIATR_RPOOL_KEY="${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_KEY_NAME}" + + # If CLOUDIATR_DIST_IMAGE is set, an image containing the dist tarballs + # has to be put in place by the user before the eviction. If it's unset, + # CLOUDIATR_DIST_DIR has to be populated before cloudiatr is executed. + CLOUDIATR_DIST_IMAGE="" + CLOUDIATR_DIST_IMAGE_SHA256="" + CLOUDIATR_DIST_DIR="/usr/electrobsd-dist" + + # Whether or not the distribution tarballs should be copied to the + # newly installed system (for example to reuse them when setting + # up jails). + CLOUDIATR_SAVE_DIST_DIR="false" + + # Default to using all the detected ada(4) devices + CLOUDIATR_DISKS="$(cloudiatr_autodetect_disks)" + + # Changing these should only be necessary if there's more + # than one disk and you don't want to a mirror. + CLOUDIATR_BPOOL_LAYOUT="default" + CLOUDIATR_RPOOL_LAYOUT="default" + + CLOUDIATR_GELI_KEY_LENGTH=256 + CLOUDIATR_GELI_EALGO=AES-XTS + # Use a passphrase for geli, this is currently interactive which isn't ideal + CLOUDIATR_GELI_USE_PASSPHRASE=false + + CLOUDIATR_SSHD_HOST_KEY_ALGORITHMS="rsa ecdsa ed25519" + + # Set to 'true' to ingore some safety-checks and increase the potential damage. + # Includes "geli kill -a" which is not limited to the disks specified above. + # Do not enable this unless the system that is being evicted doesn't contain + # any data you care about. + CLOUDIATR_MURDER_DEATH_KILL_REQUESTED=false + + # Set to 'true' to skip the image checksum check. + # "It may be insecure, but look how fast it is!" + CLOUDIATR_CHECKSUM_SMECKSUM=false + + # The config file is sourced and may overwrite any of the values above + # and most functions in this file (zogftw-style). + CLOUDIATR_CONFIG_FILE="${CLOUDIATR_CONFIG_FILE=/etc/cloudiatr.conf}" + + # A user that should be created and allowed to "su" on the new system. + CLOUDIATR_NEW_USER="cloudiatr" + + # This password is used for both root and CLOUDIATR_NEW_USER. + # + # Note that the created system will not accept root logins through + # ssh (FreeBSD default). CLOUDIATR_NEW_USER may use ssh, but has + # to use public key authentication. + CLOUDIATR_INITIAL_PASSWORD="${CLOUDIATR_NEW_USER}" + + # Local timezone. For details see tzsetup(8). + CLOUDIATR_TIME_ZONE="Europe/Berlin" + + # When set to true, cloudiatr will execute ntpdate at installtime. + # The server(s) being used depend on the install distfiles. + CLOUDIATR_USE_NTPDATE="false" + + # Any alignment should work, 1M is often recommended to prevent + # write-amplification which can result in performance degradation. + # It can also be advantageous for trimming SSDs. + CLOUDIATR_GPART_ALIGNMENT="1M" + + # Optional keyboard map for the virtual console. + # For details see kbdmap. + CLOUDIATR_KBDMAP="de.kbd" + + # Additional distributions to extract. Example: src, lib32 + CLOUDIATR_EXTRA_DISTRIBUTIONS="" + + # Value for rc.conf's rether_enable entry which controls + # whether or not MAC addresses are randomized (on ElectroBSD). + CLOUDIATR_RETHER_ENABLE="NO" + + # Set to true to not bother the user about with questions. + : "${CLOUDIATR_DONT_ASK_JUST_KISS=false}" + + # Set to true (default) to use the added swap partitions right + # after creating them. This allows installations on systems that + # have insufficient memory (512 MB, for example) and no previously + # configured swap devices. + # + # While this option is not expected to cause problems, if you + # are absolutely sure that enough memory is available you can + # disable the behaviour by setting the variable to "false". + CLOUDIATR_USE_SWAP_WHILE_INSTALLING="true" + + # Apply a workaround that is required to boot on + # Lenovo laptops like the T520 + CLOUDIATR_APPLY_LENOVO_WORKAROUND="false" + + # These subcommands are currently supported. + # The list is used by cloudiatr_show_usage() + CLOUDIATR_SUBCOMMANDS="clean-up recreate-bpool evict \ + soft-protect update update-base \ + update-bootcode update-kernel \ + version \ + ${CLOUDIATR_UNPRIVILEGED_SUBCOMMANDS}" + + # If these variables aren't set to some value, cloudiatr will abort. + # Sane values are a good idea but not mandatory. + CLOUDIATR_MANDATORY_VARIABLES="\ + CLOUDIATR_APPLY_LENOVO_WORKAROUND \ + CLOUDIATR_BPOOL_CREATE_FLAGS \ + CLOUDIATR_BPOOL_LAYOUT \ + CLOUDIATR_BPOOL_NAME \ + CLOUDIATR_BPOOL_PARTITION_SIZE \ + CLOUDIATR_CHECKSUM_SMECKSUM \ + CLOUDIATR_CONFIG_FILE \ + CLOUDIATR_DISKS \ + CLOUDIATR_DIST_DIR \ + CLOUDIATR_DONT_ASK_JUST_KISS \ + CLOUDIATR_GELI_KEY_LENGTH \ + CLOUDIATR_GELI_EALGO \ + CLOUDIATR_GELI_USE_PASSPHRASE \ + CLOUDIATR_GPART_ALIGNMENT \ + CLOUDIATR_HOSTNAME \ + CLOUDIATR_INITIAL_PASSWORD \ + CLOUDIATR_MURDER_DEATH_KILL_REQUESTED \ + CLOUDIATR_NEW_SYSTEM_DIR \ + CLOUDIATR_NEW_USER \ + CLOUDIATR_NIC \ + CLOUDIATR_USE_SWAP_WHILE_INSTALLING \ + CLOUDIATR_RETHER_ENABLE \ + CLOUDIATR_REUSE_GPART_SETUP \ + CLOUDIATR_RPOOL_CREATE_FLAGS \ + CLOUDIATR_RPOOL_KEY \ + CLOUDIATR_RPOOL_KEY_NAME \ + CLOUDIATR_RPOOL_LAYOUT \ + CLOUDIATR_RPOOL_NAME \ + CLOUDIATR_RPOOL_PARTITION_SIZE \ + CLOUDIATR_SAVE_DIST_DIR \ + CLOUDIATR_SWAP_PARTITION_SIZE \ + CLOUDIATR_TIME_ZONE \ + CLOUDIATR_USE_NTPDATE \ + CLOUDIATR_VERSION \ + " + + # These variables are allowed to be unset + CLOUDIATR_OPTIONAL_VARIABLES="\ + CLOUDIATR_DEFAULTROUTER \ + CLOUDIATR_DIST_IMAGE \ + CLOUDIATR_DIST_IMAGE_SHA256 \ + CLOUDIATR_EXTRA_DISTRIBUTIONS \ + CLOUDIATR_IP_ADDRESS \ + CLOUDIATR_KBDMAP \ + CLOUDIATR_NETMASK \ + " +} + +# Apply the workaround described at: +# https://lists.freebsd.org/pipermail/freebsd-i386/2013-March/010437.html +cloudiatr_apply_lenovo_workaround() { + local disk="${1}" \ + slice_table_original slice_table_new \ + partion_spec cylinders sectors_per_track cylinders_new heads + + if [ -z "${disk}" ]; then + cloudiatr_wtf "cloudiatr_apply_lenovo_workaround: No disk provided" + return 1 + fi + + slice_table_original=$(mktemp -t cloudiatr_slice_table_original) || return 1 + slice_table_new=$(mktemp -t cloudiatr_slice_table_new) || return 1 + + fdisk -p "${disk}" > "${slice_table_original}" || return 1 + + cylinders="$(grep '^g c' "${slice_table_original}" | cut -w -f 2 | cut -d c -f 2)" || return 1 + heads="$(grep '^g c' "${slice_table_original}" | cut -w -f 3 | cut -d h -f 2)" || return 1 + sectors_per_track="$(grep '^g c' "${slice_table_original}" | cut -w -f 4 | cut -d s -f 2)" || return 1 + + cylinders_new="$(expr "${cylinders}" \* "${sectors_per_track}")" || return 1 + + partition_spec="$(tail -n 1 ${slice_table_original})" || return 1 + + echo "# ${disk}" > "${slice_table_new}" || return 1 + echo "g c${cylinders_new} h${heads} s1" >> "${slice_table_new}" || return 1 + echo "${partition_spec}" | sed -e 's@^p 1 0xee@p 1 0x00@' >> "${slice_table_new}" || return 1 + echo "a 1" >> "${slice_table_new}" || return 1 + echo "${partition_spec}" | sed -e 's@^p 1@p 2@' >> "${slice_table_new}" || return 1 + + cloudiatr_fyi "Applying Lenovo workaround:" + diff -u "${slice_table_original}" "${slice_table_new}" || true + fdisk -f "${slice_table_new}" "${disk}" || return 1 + rm "${slice_table_original}" "${slice_table_new}" || return 1 +} + +cloudiatr_install_bootcode_on_disk() { + local boot_dir disk + + boot_dir="${1}" #/boot + disk="${2}" + + cloudiatr_fyi "Installing boot code from ${boot_dir} on ${disk} ..." + gpart bootcode -b "${boot_dir}/pmbr" -p "${boot_dir}/gptzfsboot" \ + -i "${CLOUDIATR_BOOTCODE_PARTITION}" "${disk}" || return 1 +} + +cloudiatr_install_bootcode() { + local boot_dir \ + disk + + boot_dir="${1}" + + for disk in ${CLOUDIATR_DISKS}; do + cloudiatr_install_bootcode_on_disk "${boot_dir}" "${disk}" || return 1 + done +} + +cloudiatr_gpart_disk() { + local disk="${1}" \ + disk_name + + disk_name="${disk##*/}" + + gpart create -s gpt "${disk}" || return 1 + + gpart add -s 512 -t freebsd-boot \ + -i "${CLOUDIATR_BOOTCODE_PARTITION}" "${disk}" || return 1 + gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot \ + -i "${CLOUDIATR_BOOTCODE_PARTITION}" "${disk}" || return 1 + + gpart add -s "${CLOUDIATR_BPOOL_PARTITION_SIZE}" -a "${CLOUDIATR_GPART_ALIGNMENT}" \ + -l "${CLOUDIATR_BPOOL_NAME}-${disk_name}" -t freebsd-zfs \ + -i "${CLOUDIATR_BPOOL_PARTITION}" "${disk}" || return 1 + gpart add -s "${CLOUDIATR_RPOOL_PARTITION_SIZE}" -a "${CLOUDIATR_GPART_ALIGNMENT}" \ + -l "${CLOUDIATR_RPOOL_NAME}-${disk_name}" -t freebsd-zfs \ + -i "${CLOUDIATR_RPOOL_PARTITION}" "${disk}" || return 1 + gpart add -s "${CLOUDIATR_SWAP_PARTITION_SIZE}" -a "${CLOUDIATR_GPART_ALIGNMENT}" \ + -l "swap-${disk_name}" -t freebsd-swap \ + -i "${CLOUDIATR_SWAP_PARTITION}" "${disk}" || return 1 + + # Reserve what's left for the data pool + gpart add -l "dpool-${disk_name}" -a "${CLOUDIATR_GPART_ALIGNMENT}" \ + -t freebsd-zfs -i "${CLOUDIATR_DPOOL_PARTITION}" "${disk}" || return 1 + + if "${CLOUDIATR_APPLY_LENOVO_WORKAROUND}"; then + cloudiatr_apply_lenovo_workaround "${disk}" || return 1 + fi +} + +cloudiatr_gpart_setup() { + local disks d + + disks="${*}" + + cloudiatr_fyi "Cleaning partition tables (if there are any) ..." + for d in $disks; do + gpart destroy -F "${d}" 2>/dev/null || true + done + + cloudiatr_fyi "Partitioning disks ..." + for d in $disks; do + cloudiatr_gpart_disk "${d}" || return 1 + done +} + +# Use the swap partitions on the given disks while cloudiatr is running. +# This allows to install on a system with 512MB RAM or less and no swap space. +cloudiatr_enable_swap() { + local disks d + + disks="${*}" + + cloudiatr_fyi "Using created swap space while installing ..." + for d in $disks; do + geli onetime -d "${d}p${CLOUDIATR_SWAP_PARTITION}" || return 1 + swapon "${d}p${CLOUDIATR_SWAP_PARTITION}.eli" || return 1 + done +} + +cloudiatr_disable_swap() { + local disks d + + disks="${*}" + + cloudiatr_fyi "Trying to disable previously added swap space ..." + for d in $disks; do + swapoff "${d}p${CLOUDIATR_SWAP_PARTITION}.eli" || return 1 + done +} + +cloudiatr_get_geoms() { + local postfix="${1}" \ + d geoms + + for d in ${CLOUDIATR_DISKS}; do + geom_partition="${d}${postfix}" + geoms="${geoms} ${geom_partition}" + done + echo "${geoms## }" +} + +cloudiatr_get_gpart_labels() { + local postfix="${1}" + + for d in ${CLOUDIATR_DISKS}; do + d="${d##/dev/}" + gpart show -l -p "${d}" 2>/dev/null | awk '$3 == "'"${d}${postfix}"'" {printf "%s ", $4}' + done + echo +} + +# Depends on geli being already setup +cloudiatr_create_rpool() { + local \ + rpool_elis pool_layout + + rpool_elis="$(cloudiatr_get_geoms "p${CLOUDIATR_RPOOL_PARTITION}.eli")" + pool_layout="${CLOUDIATR_RPOOL_LAYOUT}" + + if [ "${pool_layout}" = "default" ]; then + pool_layout="$(cloudiatr_get_default_pool_layout)" + fi + + cloudiatr_fyi "Creating root pool '${CLOUDIATR_RPOOL_NAME}' on ${rpool_elis}. Pool layout: ${pool_layout}" + zpool create ${CLOUDIATR_RPOOL_CREATE_FLAGS} \ + "${CLOUDIATR_RPOOL_NAME}" ${pool_layout##single-disk} $rpool_elis + + zfs set mountpoint="${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}" "${CLOUDIATR_RPOOL_NAME}" + + zfs create "${CLOUDIATR_RPOOL_NAME}/boot" + # We currently use no dedicated dataset for /etc as the kernel expects parts of it + # to be available once the rootfs has been mounted. Having two /etc's can be a bit + # of a hassle on updates and thus doesn't seem like a good default. + #zfs create -o setuid=off "${CLOUDIATR_RPOOL_NAME}/etc" + zfs create "${CLOUDIATR_RPOOL_NAME}/home" + zfs create "${CLOUDIATR_RPOOL_NAME}/home/${CLOUDIATR_NEW_USER}" + zfs create -o exec=on -o setuid=off "${CLOUDIATR_RPOOL_NAME}/tmp" + zfs create "${CLOUDIATR_RPOOL_NAME}/usr" + zfs create -o compression=gzip -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/src" + zfs create "${CLOUDIATR_RPOOL_NAME}/var" + zfs create "${CLOUDIATR_RPOOL_NAME}/usr/local" + zfs create -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/local/etc" + zfs create -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/local/src" + zfs create -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/ports" + zfs create -o compression=off -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/ports/distfiles" + zfs create -o compression=off -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/usr/ports/packages" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/cache" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/cache/pkg" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/crash" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/db" + zfs create -o exec=on -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/db/pkg" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/empty" + zfs create -o compression=gzip -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/log" + zfs create -o compression=gzip -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/mail" + zfs create -o exec=off -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/run" + zfs create -o exec=on -o setuid=off "${CLOUDIATR_RPOOL_NAME}/var/tmp" + + chmod 0750 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/crash" + chgrp mail "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/mail" + chmod 0775 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/mail" + chmod 0555 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/empty" + chflags schg,nouarch "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/empty" + chmod 1777 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/var/tmp" + chmod 1777 "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/tmp" + + zfs set readonly=on "${CLOUDIATR_RPOOL_NAME}/var/empty" +} + +cloudiatr_kernel_module_is_loaded() { + local module="${1}" + # Can't use 'kldstat -m foo' as it requires a module file on disk. WTF? + kldstat | grep -q "${module}" +} + +cloudiatr_mount_dist_image() { + # intentionally leaks non-local variable md + + if [ ! -f "${CLOUDIATR_DIST_IMAGE}" ]; then + cloudiatr_wtf "File ${CLOUDIATR_DIST_IMAGE} does not exist" + return 1 + fi + + if "${CLOUDIATR_CHECKSUM_SMECKSUM}"; then + cloudiatr_fyi "Checksum smecksum" + else + cloudiatr_fyi "Checking checksum for image file ${CLOUDIATR_DIST_IMAGE} ..." + sha256 -c "${CLOUDIATR_DIST_IMAGE_SHA256}" "${CLOUDIATR_DIST_IMAGE}" + fi + md=$(mdconfig -f "${CLOUDIATR_DIST_IMAGE}") + if [ -z "${md}" ]; then + return 1 + fi + + for potential_partition in "/dev/${md}a" "/dev/${md}p2"; do + if [ -c "${potential_partition}" ]; then + cloudiatr_fyi "Trying to mount ${potential_partition} ..." + mount -o ro "${potential_partition}" /mnt/ || return 1 + cloudiatr_fyi "Mounting ${potential_partition} worked..." + fi + done +} + +cloudiatr_extract_distribution() { + local chroot_dir \ + md extra_distribution + + chroot_dir="${1}" + + if [ -n "${CLOUDIATR_DIST_IMAGE}" ]; then + cloudiatr_mount_dist_image + else + cloudiatr_fyi "No CLOUDIATR_DIST_IMAGE specified." + cloudiatr_fyi "Using CLOUDIATR_DIST_DIR=${CLOUDIATR_DIST_DIR}!" + fi + + cloudiatr_fyi "Extracting base in ${chroot_dir} ..." + # Exclude /var/empty as it's read-only + (cd "${chroot_dir}" && tar xpf "${CLOUDIATR_DIST_DIR}/base.txz" \ + --safe-writes --exclude ./var/empty/) + + cloudiatr_fyi "Creating directories that were missing in the base tarball" + chroot "${chroot_dir}" mtree -f /etc/mtree/BSD.root.dist -d -e -u + chroot "${chroot_dir}" mtree -f /etc/mtree/BSD.var.dist -d -e -u -p var + + for extra_distribution in ${CLOUDIATR_EXTRA_DISTRIBUTIONS}; do + cloudiatr_fyi "Extracting extra distribution '${extra_distribution}'" + (cd "${chroot_dir}" && tar xpf "${CLOUDIATR_DIST_DIR}/${extra_distribution}.txz") + done + + cloudiatr_fyi "Extracting kernel (without symbols) in ${chroot_dir} ..." + (cd "${chroot_dir}" && tar xpf "${CLOUDIATR_DIST_DIR}/kernel.txz" --exclude "*.symbols") + + if [ -n "${CLOUDIATR_DIST_IMAGE}" ]; then + umount /mnt + mdconfig -d -u ${md##md} + fi +} + +cloudiatr_setup_new_user() { + local ssh_dir="${chroot_dir}/home/${CLOUDIATR_NEW_USER}/.ssh" + + cloudiatr_fyi "Creating user '${CLOUDIATR_NEW_USER}'" + + echo "${CLOUDIATR_INITIAL_PASSWORD}" | chroot "${chroot_dir}" \ + pw useradd "${CLOUDIATR_NEW_USER}" -G wheel,operator -h 0 + + # Make sure the user can login through ssh, using the + # authorized_keys file from the installation media. + # + # An extra distribution file may already have created + # the .ssh directory, so don't fail if it already exits. + mkdir -p "${ssh_dir}" + cp -v "${HOME}/.ssh/authorized_keys" "${ssh_dir}" || true + chroot "${chroot_dir}" chown -R "${CLOUDIATR_NEW_USER}" "/home/${CLOUDIATR_NEW_USER}" + chroot "${chroot_dir}" chmod -R go-rwx "/home/${CLOUDIATR_NEW_USER}" +} + +cloudiatr_create_geli_key() { + local keyfile="${1}" + + ( + umask 077 + dd bs=64 count=1 if=/dev/random of="${keyfile}" 2>/dev/null + ) +} + +cloudiatr_setup_geli() { + local disks \ + d geli_passphrase_flag + + disks="${*}" + geli_no_passphrase_flag="-P" + + cloudiatr_create_geli_key "${CLOUDIATR_RPOOL_KEY}" + + mkdir "${CLOUDIATR_NEW_SYSTEM_DIR}/geli-backups" + + for d in $disks; do + if $CLOUDIATR_GELI_USE_PASSPHRASE; then + geli_no_passphrase_flag="" + fi + cloudiatr_fyi "Initialising geli on ${d}p${CLOUDIATR_RPOOL_PARTITION} ..." + geli init -b $geli_no_passphrase_flag \ + -B "${CLOUDIATR_NEW_SYSTEM_DIR}/geli-backups/${d##/dev/}p${CLOUDIATR_RPOOL_PARTITION}.eli" \ + -K "${CLOUDIATR_RPOOL_KEY}" -l "${CLOUDIATR_GELI_KEY_LENGTH}" \ + -e "${CLOUDIATR_GELI_EALGO}" -s 4096 -V 7 "${d}p${CLOUDIATR_RPOOL_PARTITION}" > /dev/null + done + + cloudiatr_attach_geli_geoms "${CLOUDIATR_RPOOL_KEY}" "p${CLOUDIATR_RPOOL_PARTITION}" +} + +cloudiatr_attach_geli_geoms() { + local keyfile partition_id \ + disk geli_passphrase_flag + + keyfile="${1}" + partition_id="${2}" + geli_no_passphrase_flag="-p" + + for disk in ${CLOUDIATR_DISKS}; do + if $CLOUDIATR_GELI_USE_PASSPHRASE; then + geli_no_passphrase_flag="" + fi + cloudiatr_fyi "geli-attaching ${disk}${partition_id}" + geli attach $geli_no_passphrase_flag -k "$keyfile" "${disk}${partition_id}" + done +} + +cloudiatr_detach_geli_geoms() { + local partition_id + + partition_id="${1}" + + for disk in ${CLOUDIATR_DISKS}; do + cloudiatr_fyi "geli-detaching ${disk}${partition_id}" + geli detach "${disk}${partition_id}.eli" + done +} + +cloudiatr_get_bpool_geoms() { + cloudiatr_get_geoms "p${CLOUDIATR_BPOOL_PARTITION}" +} + +cloudiatr_get_disk_names() { + local disk + for disk in ${CLOUDIATR_DISKS}; do + echo "${disk##*/}" + done +} + +cloudiatr_get_number_of_disks() { + local \ + disk number_of_disks + + number_of_disks=0 + for disk in ${CLOUDIATR_DISKS}; do + number_of_disks=$((number_of_disks+1)) + done + echo "${number_of_disks}" +} + +cloudiatr_get_default_pool_layout() { + if [ "$(cloudiatr_get_number_of_disks)" = 1 ]; then + echo "single-disk" + else + echo "mirror" + fi +} + +cloudiatr_autodetect_disks() { + local \ + disk + + for disk in $(sysctl -n kern.disks); do + # Only use ada(4) devices. We obviously can't use cd(4) + # devices and using da(4) devices would require us to + # skip the one we (probably) booted from. + if [ "${disk##ada}" != "${disk}" ]; then + echo "/dev/${disk}" + fi + done +} + +cloudiatr_create_bpool() { + local \ + bpool_geoms pool_layout + + bpool_geoms="$(cloudiatr_get_bpool_geoms)" + pool_layout="${CLOUDIATR_BPOOL_LAYOUT}" + + if [ "${pool_layout}" = "default" ]; then + pool_layout="$(cloudiatr_get_default_pool_layout)" + fi + + cloudiatr_fyi "Creating boot pool '${CLOUDIATR_BPOOL_NAME}' on ${bpool_geoms}. Pool layout: ${pool_layout}" + zpool create ${CLOUDIATR_BPOOL_CREATE_FLAGS} \ + "${CLOUDIATR_BPOOL_NAME}" ${pool_layout##single-disk} \ + $bpool_geoms + + # This currently can't be set at create-time + zpool set "bootfs=${CLOUDIATR_BPOOL_NAME}" "${CLOUDIATR_BPOOL_NAME}" + + # Would be nice, but for the bootfs to work, + # its ./boot directory can't be a zfs fs. + # + # XXX: can we work around this by setting bootfs + # on bpool/boot and use a symlink from bootf/boot/boot + # to bootf/boot? + # zfs create "${CLOUDIATR_BPOOL_NAME}/boot" +} + +cloudiatr_recreate_bpool() { + local boot_dir="${1}" + + cloudiatr_create_bpool || return 1 + cloudiatr_populate_bpool "${boot_dir}" || return 1 +} + +cloudiatr_setup_tmpfs() { + mkdir -p "${CLOUDIATR_NEW_SYSTEM_DIR}" + mount -t tmpfs tmpfs "${CLOUDIATR_NEW_SYSTEM_DIR}" +} + +cloudiatr_generate_rc_conf() { + local \ + netmask + + cat < "${config_file}" +} + +cloudiatr_create_config_files() { + local chroot_dir="${1}" + + cloudiatr_generate_file loader_conf "${chroot_dir}/boot/loader.conf" + cloudiatr_generate_file sysctl_conf "${chroot_dir}/etc/sysctl.conf" + cloudiatr_generate_file rc_conf "${chroot_dir}/etc/rc.conf" + cloudiatr_generate_file fstab "${chroot_dir}/etc/fstab" + cloudiatr_generate_file resolv_conf "${chroot_dir}/etc/resolv.conf" || true +} + +cloudiatr_get_required_kernel_content() { + kldstat | awk '/k/ {print $5}' +} + +# XXX: May creates output with duplicated slashes. Ugly but harmless. +# XXX: Why do we ignore errors here? +cloudiatr_populate_bpool() { + local boot_dir \ + boot_file new_kernel_dir sub_dir bpool_mountpoint new_file + + boot_dir="${1}" + if [ "${boot_dir}" = "/" ]; then + # Prevent duplicated leading slash in log messages + boot_dir="" + fi + bpool_mountpoint="${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_BPOOL_NAME}" + + cloudiatr_fyi "Populating boot pool '${CLOUDIATR_BPOOL_NAME}' ..." + zfs set mountpoint="${bpool_mountpoint}" "${CLOUDIATR_BPOOL_NAME}" || return 1 + + # Only copy what we really need before we can import the encrypted rpool + for sub_dir in dtb firmware kernel kernel.old lua zfs modules defaults; do + mkdir -p "${bpool_mountpoint}/boot/${sub_dir}" + done + + for boot_file in $(cloudiatr_get_required_kernel_content); do + # XXX: Create missing directories here + cp -v "${boot_dir}/boot/kernel/${boot_file}" "${bpool_mountpoint}/boot/kernel/" || true + if [ -f "${boot_dir}/boot/kernel.old/${boot_file}" ]; then + cp -v "${boot_dir}/boot/kernel.old/${boot_file}" "${bpool_mountpoint}/boot/kernel.old/" || true + fi + done + + for boot_file in $(find -s "${boot_dir}/boot/" \ + -not -path "${boot_dir}/boot/kernel*" -a \ + -not -path "${boot_dir}/boot/boot*" -a \ + -not -path "${boot_dir}/boot/cdboot" -a \ + -not -path "${boot_dir}/boot/*mbr" -a \ + -type f); do + new_file="${bpool_mountpoint}/boot/${boot_file##*/boot/}" + cp -v "${boot_file}" "${new_file}" || true + done + + zfs set "mountpoint=/${CLOUDIATR_BPOOL_NAME}" "${CLOUDIATR_BPOOL_NAME}" +} + +cloudiatr_clean_up() { + cloudiatr_fyi "Exporting boot pool '${CLOUDIATR_BPOOL_NAME}' ..." + zpool export "${CLOUDIATR_BPOOL_NAME}" + cloudiatr_fyi "Exporting root pool '${CLOUDIATR_RPOOL_NAME}' ..." + zpool export "${CLOUDIATR_RPOOL_NAME}" + cloudiatr_detach_geli_geoms "p${CLOUDIATR_RPOOL_PARTITION}" + umount "${CLOUDIATR_NEW_SYSTEM_DIR}" + rmdir "${CLOUDIATR_NEW_SYSTEM_DIR}" + if "${CLOUDIATR_USE_SWAP_WHILE_INSTALLING}"; then + if ! cloudiatr_disable_swap ${CLOUDIATR_DISKS}; then + cloudiatr_fyi "Failed to remove all the added swap space." + cloudiatr_fyi "If the system is low on memory the problem can be safely ignored." + fi + fi +} + +cloudiatr_generate_ssh_hostkeys() { + local chroot_dir \ + real_hostname + + chroot_dir="${1}" + real_hostname="$(hostname)" + + hostname "${CLOUDIATR_HOSTNAME}" + cloudiatr_fyi "Generating ssh host keys for ${CLOUDIATR_HOSTNAME} ..." + for key_alg in ${CLOUDIATR_SSHD_HOST_KEY_ALGORITHMS}; do + key_file="${chroot_dir}/etc/ssh/ssh_host_${key_alg}_key" + if ! ssh-keygen -q -t "${key_alg}" -f "${key_file}" -N ""; then + if [ "${key_alg}" = "ed25519" ]; then + # ed25519 isn't supported on FreeBSD 10.0 and earlier, + # thus we allow this to fail + continue + fi + return 1 + fi + ssh-keygen -l -v -f "${key_file}.pub" + done + hostname "${real_hostname}" +} + +cloudiatr_collect_evidence() { + local \ + evidence_dataset evidence disk_name + + evidence_dataset="${CLOUDIATR_RPOOL_NAME}/cloudiatr-evidence" + + cloudiatr_fyi "Collecting 'evidence' in /${evidence_dataset} ..." + zfs create "${evidence_dataset}" + for evidence in "${CLOUDIATR_CONFIG_FILE}" "${0}" \ + "${CLOUDIATR_NEW_SYSTEM_DIR}/geli-backups/"* \ + "${CLOUDIATR_RPOOL_KEY}"; do + cp -p "${evidence}" "${CLOUDIATR_NEW_SYSTEM_DIR}/${evidence_dataset}" + done + + for disk_name in $(cloudiatr_get_disk_names); do + evidence="${CLOUDIATR_NEW_SYSTEM_DIR}/${evidence_dataset}/${disk_name}.gpart" + gpart backup "${disk_name}" > "${evidence}" + done + + if "${CLOUDIATR_SAVE_DIST_DIR}"; then + cp -r "${CLOUDIATR_DIST_DIR%%/}" "${CLOUDIATR_NEW_SYSTEM_DIR}/${evidence_dataset}/" + fi + + find "${CLOUDIATR_NEW_SYSTEM_DIR}/${evidence_dataset}" -type f | sort +} + +cloudiatr_generate_sshd_config_extension() { + cat <> ${chroot_dir}/etc/ssh/sshd_config + + if "${CLOUDIATR_USE_NTPDATE}"; then + chroot ${chroot_dir} service ntpdate onestart || true + fi + + umount ${chroot_dir}/dev/ + + cloudiatr_setup_new_user + + # Copying the zpool.cache is no longer necessary on ElectroBSD + # and recent FreeBSD versions, but doesn't hurt. + cp /boot/zfs/zpool.cache "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}/boot/zfs/zpool.cache" + + cloudiatr_populate_bpool "${CLOUDIATR_NEW_SYSTEM_DIR}/${CLOUDIATR_RPOOL_NAME}" + + cloudiatr_collect_evidence + + cloudiatr_fyi "Setting final mountpoints on root pool '${CLOUDIATR_RPOOL_NAME}' ..." + zfs umount "${CLOUDIATR_RPOOL_NAME}" + zfs set mountpoint=legacy "${CLOUDIATR_RPOOL_NAME}" + for fs in boot home tmp usr var; do + zfs set "mountpoint=/${fs}" "${CLOUDIATR_RPOOL_NAME}/${fs}" + done + + cloudiatr_clean_up +} + +# The murder-death-kill feature was added for testing. After the introduction of +# the boring "clean-up" subcommand (which doesn't involve killing) it could be +# considered obsolete, but keeping it around makes cloudiatr more awesome. +cloudiatr_murder_death_kill() { + if "${CLOUDIATR_MURDER_DEATH_KILL_REQUESTED}"; then + cloudiatr_fyi "You really asked for it. Murder death kill in progress ..." + zpool export "${CLOUDIATR_BPOOL_NAME}" || true + zpool export "${CLOUDIATR_RPOOL_NAME}" || true + umount "${CLOUDIATR_NEW_SYSTEM_DIR}" || true + geli kill -a || true + fi +} + +cloudiatr_request_consent() { + local message="${*}" \ + response + + if "${CLOUDIATR_DONT_ASK_JUST_KISS}"; then + # ... the data goodbye. + return 0 + fi + + echo -n "cloudiatr: ${message} [y/n] " + # XXX: Don't use "read -p" as it may work unreliably + read response + + # "No" means "no". Everything but "y" also means "no". + [ "${response}" = "y" ] +} + +cloudiatr_has_eviction_consent() { + echo "cloudiatr (${CLOUDIATR_VERSION}) can't wait to evict '$(hostname)' ..." + echo + echo "Depending on your jurisdiction, 'eviction without consent' may be against the law." + echo "cloudiatr doesn't bother to make backups of the existing data. That's what zogftw is for." + echo + cloudiatr_request_consent "Continue eviction?" +} + +cloudiatr_evict() { + if cloudiatr_has_eviction_consent; then + if "${CLOUDIATR_MURDER_DEATH_KILL_REQUESTED}"; then + cloudiatr_murder_death_kill + else + cloudiatr_fyi "You asked for it ..." + fi + cloudiatr_fyi "Eviction in progress ..." + cloudiatr_evict_local_system + cloudiatr_fyi "Looks like somebody managed to install a real operating system ..." + if cloudiatr_request_consent "Reboot now?"; then + shutdown -r now + fi + return 0 + else + cloudiatr_fyi "Eviction aborted in time ..." + return 1 + fi +} + +cloudiatr_has_soft_protect_consent() { + cloudiatr_request_consent "Put $(uname) in 'Soviet Germany' mode?" +} + +# XXX: Only works for the cloudiatr disk layout. +cloudiatr_soft_protect() { + local \ + mirror_name device_to_clear number_of_disks + + mirror_name="vdev-remains" + number_of_disks="$(cloudiatr_get_number_of_disks)" + + cloudiatr_fyi "Destroying ${CLOUDIATR_BPOOL_NAME} ..." + cloudiatr_fyi "Use 'geli kill -a' to 'hard-protect' your data right now. No recovery without remote backups!" + + if zpool list "${CLOUDIATR_BPOOL_NAME}" >/dev/null 2>&1; then + zpool destroy "${CLOUDIATR_BPOOL_NAME}" || true + fi + + if [ "${number_of_disks}" = 1 ]; then + cloudiatr_fyi "Nuking former ${CLOUDIATR_BPOOL_NAME} vdev from orbit ..." + device_to_clear="$(cloudiatr_get_bpool_geoms)" + else + cloudiatr_fyi "Nuking former ${CLOUDIATR_BPOOL_NAME} vdevs from orbit using gmirror power ..." + gmirror load 2>/dev/null || true + gmirror label "${mirror_name}" $(cloudiatr_get_bpool_geoms) + device_to_clear="/dev/mirror/${mirror_name}" + fi + + geli onetime -s 4096 -e "${CLOUDIATR_GELI_EALGO}" "${device_to_clear}" + dd if=/dev/zero bs=1M of="${device_to_clear}.eli" 2>/dev/null || true + geli detach "${device_to_clear##/dev/}" + + cloudiatr_fyi "Done. Levelling nuked wasteland with zeroes ..." + dd if=/dev/zero bs=1M of="${device_to_clear}" 2>/dev/null || true + + if [ "${number_of_disks}" != 1 ]; then + gmirror destroy "${mirror_name}" + fi + cloudiatr_fyi "Done. $(uname) should remain working as expected until the next shutdown ..." + cloudiatr_fyi "Remember to 'unprotect' the system before consensual reboots (or use the opportunity to test your backup system)" +} + +cloudiatr_check_privileges() { + local subcommand \ + unprivileged_subcommand uid user + + subcommand="${1}" + + for unprivileged_subcommand in ${CLOUDIATR_UNPRIVILEGED_SUBCOMMANDS}; do + if [ "${subcommand}" != "${unprivileged_subcommand}" ]; then + # subcommand does not require privileges + return 0 + fi + done + + uid="$(id -u)" + user="$(id -un)" + + if [ "${uid}" != 0 ]; then + cloudiatr_wtf "Check your privileges, $user. It looks like you might not have enough of them!" + cloudiatr_fyi "Hint: Using 'sudo' or 'su' might help." + return 1 + fi + + return 0 +} + +cloudiatr_usage() { + local \ + subcommand + + for subcommand in ${CLOUDIATR_SUBCOMMANDS}; do + echo "cloudiatr [-f config-file] $subcommand" + done +} + +cloudiatr_update_base() { + local \ + base_dist base_dbg_dist + + base_dist="${CLOUDIATR_DIST_DIR}/base.txz" + base_dbg_dist="${CLOUDIATR_DIST_DIR}/base-dbg.txz" + + if [ ! -f "${base_dist}" ]; then + cloudiatr_wtf "Base update impossible. '${base_dist}' does not exist" + return 1 + fi + cloudiatr_fyi "Updating base system ..." + chflags -R noschg /bin/ /lib/ /sbin/ /usr/bin/ /usr/lib /libexec/ || return 1 + cd / || return 1 + tar xpf "${base_dist}" \ + --exclude ./etc/ --exclude ./var/empty --exclude ./usr/src || return 1 + if [ -f "${base_dbg_dist}" ]; then + tar xpf "${base_dbg_dist}" || return 1 + fi +} + +cloudiatr_update_kernel() { + local \ + kernel_dist kernel_dbg_dist \ + kernel_dir kernel_dir_old \ + kernel_dbg_dir kernel_dbg_dir_old \ + + kernel_dist="${CLOUDIATR_DIST_DIR}/kernel.txz" + kernel_dbg_dist="${CLOUDIATR_DIST_DIR}/kernel-dbg.txz" + kernel_dir=/boot/kernel + kernel_dir_old=/boot/kernel.old + kernel_dbg_dir="/usr/lib/debug${kernel_dir}" + kernel_dbg_dir_old="/usr/lib/debug${kernel_dir_old}" + + if [ ! -f "${kernel_dist}" ]; then + cloudiatr_wtf "Kernel update impossible. '${kernel_dist}' does not exist" + return 1 + fi + cloudiatr_fyi "Updating kernel ..." + cd / || return 1 + if [ -d "${kernel_dir_old}" ]; then + cloudiatr_fyi "Removing ${kernel_dir_old}" + rm -r "${kernel_dir_old}" || return 1 + fi + cloudiatr_fyi "Moving ${kernel_dir} to ${kernel_dir_old} ..." + mv "${kernel_dir}" "${kernel_dir_old}" || return 1 + + cloudiatr_fyi "Extracting ${kernel_dist} ..." + tar xpf "${kernel_dist}" || return 1 + + if [ -d "${kernel_dbg_dir_old}" ]; then + cloudiatr_fyi "Removing ${kernel_dbg_dir_old}" + rm -r "${kernel_dbg_dir_old}" || return 1 + fi + if [ -d "${kernel_dbg_dir}" ]; then + cloudiatr_fyi "Moving ${kernel_dbg_dir} to ${kernel_dbg_dir_old} ..." + mv "${kernel_dbg_dir}" "${kernel_dbg_dir_old}" || return 1 + fi + cloudiatr_fyi "Extracting ${kernel_dbg_dist} ..." + if [ -f "${kernel_dbg_dist}" ]; then + tar xpf "${kernel_dbg_dist}" || return 1 + fi +} + +cloudiatr_show_version() { + echo "cloudiatr ${CLOUDIATR_VERSION}" +} + +cloudiatr_main() { + local mode + + if [ "${1}" = "-f" ]; then + shift + CLOUDIATR_CONFIG_FILE="${1}" + shift + if [ -z "${CLOUDIATR_CONFIG_FILE}" ] || + ! [ -f "${CLOUDIATR_CONFIG_FILE}" ]; then + cloudiatr_wtf "No existing config file specified." + return 1 + fi + fi + + mode="${1}" + + cloudiatr_check_privileges "${mode}" || return 1 + + cloudiatr_init "${mode}" + + case "${mode}" in + clean-up) + set +e + cloudiatr_clean_up + ;; + cmd) + shift + "${@}" + ;; + recreate-bpool) + cloudiatr_recreate_bpool "/" + ;; + evict) + cloudiatr_evict + ;; + soft-protect) + if cloudiatr_has_soft_protect_consent; then + cloudiatr_soft_protect + fi + ;; + update) + cloudiatr_update_base || return 1 + cloudiatr_install_bootcode "/boot" || return 1 + cloudiatr_update_kernel || return 1 + cloudiatr_create_bpool || return 1 + cloudiatr_populate_bpool "/" || return 1 + ;; + update-base) + cloudiatr_update_base || return 1 + ;; + update-bootcode) + cloudiatr_install_bootcode "/boot" || return 1 + ;; + update-kernel) + cloudiatr_update_kernel || return 1 + ;; + show-config) + cloudiatr_show_config + ;; + version) + cloudiatr_show_version + ;; + #rekey) + # XXX: Not yet implemented + *) + cloudiatr_usage + cloudiatr_wtf "Invalid or missing subcommand" + return 1 + ;; + esac + +} + +cloudiatr_main "${@}" -- 2.32.0 From f06f3f06adcecee38b58bf1c987618622fbca908 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 10 Aug 2017 14:20:25 +0200 Subject: [PATCH 177/325] geom_nop(4): Include BIO_ZONE commands in the statistics Obtained from: ElectroBSD --- sys/geom/nop/g_nop.c | 6 ++++++ sys/geom/nop/g_nop.h | 1 + 2 files changed, 7 insertions(+) diff --git a/sys/geom/nop/g_nop.c b/sys/geom/nop/g_nop.c index 238a3f6b753c..cd0e3983e628 100644 --- a/sys/geom/nop/g_nop.c +++ b/sys/geom/nop/g_nop.c @@ -153,6 +153,9 @@ g_nop_start(struct bio *bp) case BIO_CMD2: sc->sc_cmd2s++; break; + case BIO_ZONE: + sc->sc_zone_cmds++; + break; } mtx_unlock(&sc->sc_lock); if (failprob > 0) { @@ -282,6 +285,7 @@ g_nop_create(struct gctl_req *req, struct g_class *mp, struct g_provider *pp, sc->sc_cmd0s = 0; sc->sc_cmd1s = 0; sc->sc_cmd2s = 0; + sc->sc_zone_cmds = 0; sc->sc_readbytes = 0; sc->sc_wrotebytes = 0; mtx_init(&sc->sc_lock, "gnop lock", NULL, MTX_DEF); @@ -659,6 +663,7 @@ g_nop_ctl_reset(struct gctl_req *req, struct g_class *mp) sc->sc_cmd0s = 0; sc->sc_cmd1s = 0; sc->sc_cmd2s = 0; + sc->sc_zone_cmds = 0; sc->sc_readbytes = 0; sc->sc_wrotebytes = 0; } @@ -722,6 +727,7 @@ g_nop_dumpconf(struct sbuf *sb, const char *indent, struct g_geom *gp, sbuf_printf(sb, "%s%ju\n", indent, sc->sc_cmd0s); sbuf_printf(sb, "%s%ju\n", indent, sc->sc_cmd1s); sbuf_printf(sb, "%s%ju\n", indent, sc->sc_cmd2s); + sbuf_printf(sb, "%s%ju\n", indent, sc->sc_zone_cmds); sbuf_printf(sb, "%s%ju\n", indent, sc->sc_readbytes); sbuf_printf(sb, "%s%ju\n", indent, diff --git a/sys/geom/nop/g_nop.h b/sys/geom/nop/g_nop.h index 353e5c5b8c09..66d7d2010277 100644 --- a/sys/geom/nop/g_nop.h +++ b/sys/geom/nop/g_nop.h @@ -78,6 +78,7 @@ struct g_nop_softc { uintmax_t sc_cmd0s; uintmax_t sc_cmd1s; uintmax_t sc_cmd2s; + uintmax_t sc_zone_cmds; uintmax_t sc_readbytes; uintmax_t sc_wrotebytes; char* sc_physpath; -- 2.32.0 From fb85ef857356b00a7408963b515166ada092d852 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 11 Aug 2017 16:40:28 +0200 Subject: [PATCH 178/325] strip-freebsd.sh: Delete sendmail rc scripts as we don't ship the license either Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index c6875e6baab8..e5287317b07a 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -140,6 +140,14 @@ get_directories_to_ditch() { echo "${potential_directory}" fi done + + # These files aren't useful without sendmail itself. + # More importantly, some of them are licensed under the + # "Sendmail License" which isn't part of the directory. + # + # Deleting them prevents them from ending up in + # the source tarball without license text. + echo etc/sendmail } purify_cwd() { -- 2.32.0 From dc169dea9099bc499c5161f0aefac6e7730e86a5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 11 Aug 2017 17:41:16 +0200 Subject: [PATCH 179/325] release/scripts/strip-freebsd.sh: Ditch share/doc Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index e5287317b07a..d74f5b4c2d24 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -148,6 +148,10 @@ get_directories_to_ditch() { # Deleting them prevents them from ending up in # the source tarball without license text. echo etc/sendmail + + # Remove the "4.4BSD legacy docs". The content isn't + # needed and the license terms aren't obvious. + echo share/doc } purify_cwd() { -- 2.32.0 From d7452f3b2c59e5ad1f73fa88026003127a46d530 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 11 Aug 2017 20:53:54 +0200 Subject: [PATCH 180/325] share/mk/src.opts.mk: Disable SHAREDOCS to unbreak the build src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 0117d6594697..5e4bd3128e65 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -150,7 +150,6 @@ __DEFAULT_YES_OPTIONS = \ RCS \ RESCUE \ SETUID_LOGIN \ - SHAREDOCS \ SYSCONS \ SYSTEM_COMPILER \ SYSTEM_LINKER \ @@ -246,6 +245,7 @@ __DEFAULT_NO_OPTIONS += \ RCMDS \ SENDMAIL \ SVNLITE \ + SHAREDOCS \ SOURCELESS \ SOURCELESS_HOST \ SOURCELESS_UCODE \ -- 2.32.0 From 4d4ce376684c35131638b358a87a9b3411ec1d7c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 21 Aug 2017 18:56:06 +0200 Subject: [PATCH 181/325] share/mk: Disable AMD by default It's obsolete and already deprecated by upstream in favour of autofs src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 5e4bd3128e65..15faff8475dc 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -55,7 +55,6 @@ ____: __DEFAULT_YES_OPTIONS = \ ACCT \ ACPI \ - AMD \ APM \ AT \ AUDIT \ @@ -223,6 +222,7 @@ __DEFAULT_YES_OPTIONS += \ # Disable a bunch of additional options that default to yes in FreeBSD __DEFAULT_NO_OPTIONS += \ ATM \ + AMD \ BLUETOOTH \ BSDINSTALL \ CCD \ -- 2.32.0 From 53eecb80818b8fa5ded12758b923a5fbd69ba674 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 12:05:18 +0200 Subject: [PATCH 182/325] share/mk/src.opts.mk: Disable NETGRAPH by default src.conf(5) is regenerated in a seperate commit to reduce the chances of merge conflicts when rebasing. Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 15faff8475dc..fc34b3c072f5 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -133,7 +133,6 @@ __DEFAULT_YES_OPTIONS = \ MALLOC_PRODUCTION \ MANDOCDB \ NETCAT \ - NETGRAPH \ NLS_CATALOGS \ NS_CACHING \ NTP \ @@ -236,6 +235,7 @@ __DEFAULT_NO_OPTIONS += \ IPFILTER \ ISCSI \ LIB32 \ + NETGRAPH \ NDIS \ RBOOTD \ PC_SYSINSTALL \ -- 2.32.0 From f18f24a6a2de335b18552ba984a2f40653bcb531 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 21 Aug 2017 19:05:13 +0200 Subject: [PATCH 183/325] strip-freebsd.sh: Add amd to the list of contrib directories to ditch Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index d74f5b4c2d24..75b48141a6d9 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -94,7 +94,7 @@ get_unused_contrib_dirs() { # XXX: gcc can't be deleted because parts of it are apparently # required to build libc. This should be investigated more thoroughly, # hopefully it can be fixed. - echo "apr apr-util ipfilter ofed sendmail serf subversion tcsh" + echo "amd apr apr-util ipfilter ofed sendmail serf subversion tcsh" } # Only includes directory that aren't architecture-specific -- 2.32.0 From 9e318874d816a10141029bcad7551e93c65cef8f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 21 Aug 2017 19:18:58 +0200 Subject: [PATCH 184/325] strip-freebsd.sh: Try to remove directories that reference unused contrib code Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 75b48141a6d9..b8be6b84ec9d 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -134,13 +134,24 @@ get_directories_to_ditch() { echo "sys/dev/${sys_dev}" done + # For each known unused directory remove it from + # "contrib" (where most of the code is located) + # and "usr.sbin" and "usr.bin" if a directory with + # the same name exists. for contrib_dir in $(get_unused_contrib_dirs); do - potential_directory="contrib/${contrib_dir}" + for potential_parent in contrib usr.sbin usr.bin; do + potential_directory="${potential_parent}/${contrib_dir}" if [ -d "${potential_directory}" ]; then echo "${potential_directory}" fi + done done + # Not automatically detected in the loop above + # because the contrib directories are named + # differently. + echo "usr.bin/svn" + # These files aren't useful without sendmail itself. # More importantly, some of them are licensed under the # "Sendmail License" which isn't part of the directory. -- 2.32.0 From d72af1f07d2db0333b837ee2ee4a724a001da954 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 04:38:24 +0200 Subject: [PATCH 185/325] release/scripts/strip-freebsd.sh: Ditch the liquidio firmware ... which was added in r323509/5543e587c7a (not yet MFC'd) with unclear license conditions. A modified version of the supplised MK_SOURCELESS_UCODE fix from https://lists.freebsd.org/pipermail/svn-src-all/2017-September/151110.html was committed in r323543/0091cace28a, but the question about the license remains without reply (2017-09-18). Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index b8be6b84ec9d..a394bb16792f 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -79,7 +79,7 @@ get_unsupported_architectures() { # These depend on or contain proprietary firmware that is included in sys/contrib/dev get_tainted_sys_contrib_devs() { - echo "drm2 ipw iwi iwm iwn mwl npe otus ral rsu rtwn run uath urtwn wpi" + echo "drm2 ipw iwi iwm iwn liquidio mwl npe otus ral rsu rtwn run uath urtwn wpi" } # These require proprietary firmware that is included in sys/dev -- 2.32.0 From d729e076bbe68058994ac0f190b42c99a68c9ccc Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 15 Sep 2017 13:17:05 +0200 Subject: [PATCH 186/325] release/scripts/strip-freebsd.sh: Add tdfx to the list of tainted sys devs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TL;DR: gesehen, gelacht, gelöscht Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index a394bb16792f..4b15c8e87f99 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -88,6 +88,20 @@ get_tainted_sys_devs() { # bce has already been taken care of by get_bce_files() above echo "bxe ctau cx cxgb cxgbe ispfw qlxgbe qlnx" \ "it tw" + # known tdfx issues: + # tdfx was supposedly "developed by Coleman Kane" but the license + # text mentions "Gardner Buchanan" in clause 3 and 4 without + # a matching "Copyright" line. Did somebody copy and paste the + # license text from sys/dev/sn without reading it? + # + # Also one of tdfx's header files "is basically a derivative of + # some sys/dhio.h file" which indicates that "some" other file's + # license may be relevant here. + # + # Luckily "3Dfx Voodoo" support isn't too relevant for most ElectroBSD + # use cases so figuring out the license problems is a low priority issue + # because we can simply ditch the offending code. + echo "tdfx" } get_unused_contrib_dirs() { -- 2.32.0 From 9037b91365c88f4d0447b09a4d6e54b27f2ed51c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 23 Aug 2017 08:50:43 +0200 Subject: [PATCH 187/325] share/dtrace: Import bio-request-latency Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/bio-request-latency | 116 +++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+) create mode 100755 share/dtrace/bio-request-latency diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 4386cba69afe..66d0639ba365 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -6,6 +6,7 @@ .include SCRIPTS= blocking \ + bio-request-latency \ disklatency \ disklatencycmd \ fbt-time \ diff --git a/share/dtrace/bio-request-latency b/share/dtrace/bio-request-latency new file mode 100755 index 000000000000..be107a88edf2 --- /dev/null +++ b/share/dtrace/bio-request-latency @@ -0,0 +1,116 @@ +#!/usr/sbin/dtrace -s +/*************************************************************************** + * bio-request-latency.d + * + * Shows the latency for each disk and BIO type. + * + * io probes without BIO are ignored as they don't provide + * enough information. For details see: + * https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220965 + * + * Inspired by: + * http://dtrace.org/blogs/ahl/2014/08/31/openzfs-tuning/ + * + * Copyright (c) 2017 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option dynvarsize=10m + +BEGIN +{ + start = timestamp; + bogus_timestamps = 0; + + bio_command[0x01] = "BIO_READ"; + bio_command[0x02] = "BIO_WRITE"; + bio_command[0x03] = "BIO_DELETE"; + bio_command[0x04] = "BIO_GETATTR"; + bio_command[0x05] = "BIO_FLUSH"; + bio_command[0x06] = "BIO_CMD0"; + bio_command[0x07] = "BIO_CMD1"; + bio_command[0x08] = "BIO_CMD2"; + bio_command[0x09] = "BIO_ZONE"; +} + +io:::start +/arg0 != NULL/ +{ + io_start[arg0] = timestamp; +} + +/* Cache timestamp so we can sanity check it */ +io:::done +{ + this->timestamp = timestamp; +} + +io:::done +/arg0 != NULL && io_start[arg0] && io_start[arg0] > this->timestamp/ +{ + /* + * At least in theory the timestamps goes up, in practice + * timestamps can jump back due to bugs like: + * https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218452 + * + * If this happens the timestamp delta is obviously useless + * for our purposes so lets not include it in the stats. + */ + bogus_timestamps++; + io_start[arg0] = 0; +} + +/* If this section gets entered this->timestamp isn't obviouly bogus */ +io:::done +/arg0 != NULL && io_start[arg0]/ +{ + this->delta = (this->timestamp - io_start[arg0]) / 1000; + this->bio_command = (bio_command[args[0]->bio_cmd] != NULL) ? + bio_command[args[0]->bio_cmd] : "Unregistered command"; + this->disk_name = stringof(args[0]->bio_disk->d_geom->name); + + @quantize[this->disk_name, this->bio_command] = quantize(this->delta); + @average[this->disk_name, this->bio_command] = avg(this->delta); + @stddev[this->disk_name, this->bio_command] = stddev(this->delta); + @max[this->disk_name, this->bio_command] = max(this->delta); + @min[this->disk_name, this->bio_command] = min(this->delta); + @requests[this->disk_name, this->bio_command] = count(); + @ios[this->disk_name, this->bio_command] = count(); + @throughput[this->disk_name, this->bio_command] = sum(args[0]->bio_length); + + io_start[arg0] = 0; +} + +END +{ + printa(@quantize); + + normalize(@ios, (timestamp - start) / 1000000000); + normalize(@throughput, (timestamp - start) / 1000000000 * 1024); + + printf("%-8s %-12s %10s | %-15s %-23s | %6s %12s\n", + "Disk", "Bio Type", "Requests", "", "Latency in us", "iops", "Throughput"); + printf("%-32s | %9s %9s %9s %9s |\n", "", "avg", "stddev", "max", "min"); + printa("%-8s %-12s %@10u | %@9u %@9u %@9u %@9u | %@6u %@8u k/s\n", + @requests, @average, @stddev, @max, @min, @ios, @throughput); +} + +END +/bogus_timestamps != 0/ +{ + printf("\nLatency for %d requests not measured due to obviously bogus timestamps.\n", + bogus_timestamps); + printf("Note that some of the other measurements may be based on less-obviously bogus timestamps.\n"); +} -- 2.32.0 From 1084428e45b3017a88bd3e8d3837a54e5783e935 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 23 Aug 2017 08:56:36 +0200 Subject: [PATCH 188/325] share/dtrace: Import zfs-txg-sync-info 2017-08-23-ed990899 Obtained from: ElectroBSD --- share/dtrace/Makefile | 3 +- share/dtrace/zfs-txg-sync-info | 58 ++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 1 deletion(-) create mode 100755 share/dtrace/zfs-txg-sync-info diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 66d0639ba365..218da556452f 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -23,7 +23,8 @@ SCRIPTS= blocking \ tcpstate \ tcptrack \ udptrack \ - zfsdbg-msg + zfsdbg-msg \ + zfs-txg-sync-info SCRIPTSDIR= ${SHAREDIR}/dtrace diff --git a/share/dtrace/zfs-txg-sync-info b/share/dtrace/zfs-txg-sync-info new file mode 100755 index 000000000000..00f5614cac2c --- /dev/null +++ b/share/dtrace/zfs-txg-sync-info @@ -0,0 +1,58 @@ +#!/usr/sbin/dtrace -s +/*************************************************************************** + * zfs-txg-sync-info.d + * + * Shows the txg sync time and the amount of dirty data that got synced. + * By default all pools are monitored. If a pool name is specified as + * first argument, only that pool is monitored. + * + * Inspired by: + * http://dtrace.org/blogs/ahl/2014/08/31/openzfs-tuning/ + * + * Copyright (c) 2017 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet +#pragma D option defaultargs + +BEGIN +{ + pool_to_watch = $$1; +} + +txg-syncing +/pool_to_watch == NULL || pool_to_watch == ((dsl_pool_t *)arg0)->dp_spa->spa_name/ +{ + this->pool_name = stringof(((dsl_pool_t *)arg0)->dp_spa->spa_name); + sync_start[this->pool_name] = timestamp; + dp_dirty_total[this->pool_name] = ((dsl_pool_t *)arg0)->dp_dirty_total; + dp_dirty_percentage[this->pool_name] = dp_dirty_total[this->pool_name] / (`zfs_dirty_data_max / 100); +} + +txg-synced +/this->pool_name != NULL/ +{ + this->sync_time = timestamp - sync_start[this->pool_name]; + + printf("%Y: %s txg written in %d.%02d seconds. %3d.%02d MB were dirty (%2d%% of %d MB)\n", + walltimestamp, + this->pool_name, + this->sync_time / 1000000000, + this->sync_time / 10000000 % 100, + dp_dirty_total[this->pool_name] / 1024 / 1024, + dp_dirty_total[this->pool_name] / 1024 % 100, + dp_dirty_percentage[this->pool_name], + `zfs_dirty_data_max / 1024 / 1024); +} -- 2.32.0 From 9d3a3c02a79326c28e44419299f9ac0d8ca96c16 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 25 Aug 2017 10:30:37 +0200 Subject: [PATCH 189/325] Add share/dtrace/iosched-monitor (unfinished WIP) so it no longer gets listed as 'untracked' Obtained from: ElectroBSD --- share/dtrace/iosched-monitor | 87 ++++++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100755 share/dtrace/iosched-monitor diff --git a/share/dtrace/iosched-monitor b/share/dtrace/iosched-monitor new file mode 100755 index 000000000000..dca024a49dc7 --- /dev/null +++ b/share/dtrace/iosched-monitor @@ -0,0 +1,87 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * iosched-monitor + * + * Monitors the behavior of the Dynamic I/O scheduler. + * + * Copyright (c) 2017 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option quiet + +dtrace:::BEGIN +{ + bio_command[0x01] = "BIO_READ"; + bio_command[0x02] = "BIO_WRITE"; + bio_command[0x03] = "BIO_DELETE"; + bio_command[0x04] = "BIO_GETATTR"; + bio_command[0x05] = "BIO_FLUSH"; + bio_command[0x06] = "BIO_CMD0"; + bio_command[0x07] = "BIO_CMD1"; + bio_command[0x08] = "BIO_CMD2"; + bio_command[0x09] = "BIO_ZONE"; +} + +fbt::cam_iosched_bio_complete:entry +{ + this->isc = args[0]; + this->bp = args[1]; + + this->bio_command = (bio_command[this->bp->bio_cmd] != NULL) ? + bio_command[this->bp->bio_cmd] : "Unregistered command"; +} + +fbt::cam_iosched_bio_complete:return +/this->isc && this->bio_command == "BIO_DELETE"/ +{ + printf("%Y: cam_iosched_bio_complete returned for BIO_DELETE. Pending trims: %d\n", + walltimestamp, this->isc->trim_stats.pending); +} + +fbt::cam_iosched_trim_done:entry +{ + this->isc = args[0]; + printf("%Y: cam_iosched_trim_done called. Pending trims: %d\n", + walltimestamp, this->isc->trim_stats.pending); +} + +fbt::cam_iosched_put_back_trim:entry +{ + this->isc = args[0]; +} + +fbt::cam_iosched_put_back_trim:return +/this->isc/ +{ + printf("%Y: cam_iosched_put_back returned. Pending trims: %d\n", + walltimestamp, this->isc->trim_stats.pending); +} + +fbt::cam_iosched_next_trim:entry +{ + this->isc = args[0]; +} + +fbt::cam_iosched_next_trim:return +/this->isc/ +{ + printf("%Y: cam_iosched_next_trim returned. Pending trims: %d\n", + walltimestamp, this->isc->trim_stats.pending); +} + +END +{ +} -- 2.32.0 From ad83098edb01c4ad2043f3a689e94e6322714727 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 27 Aug 2017 18:44:43 +0200 Subject: [PATCH 190/325] share/dtrace: Import flowtrace 2017-01-28/d600753af03 Obtained from: ElectroBSD --- share/dtrace/Makefile | 1 + share/dtrace/flowtrace | 165 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 166 insertions(+) create mode 100755 share/dtrace/flowtrace diff --git a/share/dtrace/Makefile b/share/dtrace/Makefile index 218da556452f..b94fb50f40fd 100644 --- a/share/dtrace/Makefile +++ b/share/dtrace/Makefile @@ -10,6 +10,7 @@ SCRIPTS= blocking \ disklatency \ disklatencycmd \ fbt-time \ + flowtrace \ geli-key-monitor \ geli-request-monitor \ hotopen \ diff --git a/share/dtrace/flowtrace b/share/dtrace/flowtrace new file mode 100755 index 000000000000..9e890f61a311 --- /dev/null +++ b/share/dtrace/flowtrace @@ -0,0 +1,165 @@ +#!/usr/sbin/dtrace -s + +/*************************************************************************** + * flowtrace.d + * + * Enables flow tracing after entering the function specified as + * first argument and disables it again after the function returns. + * + * For a number of currently-hardcoded functions the children + * don't get shown in the output to make it easier to comprehend. + * + * XXX: It would be more efficient to not generate probes for them. + * This probably requires another script that generates the D + * code on the fly. + * + * Copyright (c) 2012 Fabian Keil + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + ***************************************************************************/ + +#pragma D option defaultargs +#pragma D option dynvarsize=10m +#pragma D option bufsize=50m +#pragma D option switchrate=10hz +#pragma D option quiet + +dtrace:::BEGIN +{ + /* + * Initialize indent[] which is used to indent the probefunc + * according to the stackdepth. Unlike "#pragma D option flowindent" + * this doesn't break in case of missing return probes. + * + * The rest of the values are filled in the dtrace::BEGIN probe + * at the end of the script. + */ + indent[0] = ""; + + traced_function = $$1; + + this->timestamp = walltimestamp; + this->msecs = (this->timestamp / 1000000) % 1000; + printf("%d %d %Y.%.3d 00: Trace in progress. Waiting to enter %s. Hit CTRL-C to exit.\n", + cpu, tid, this->timestamp, this->msecs, traced_function); +} + +/* Prepare to trace every function on this thread from now on ... */ +fbt::$$1:entry +{ + self->trace_me_yo = 1; +} + +/* ... except for the children of these functions. */ +fbt::hpet_intr:return, +fbt::lapic_handle_intr:return, +fbt::termcn_cnputc:return, +fbt::printf:return, +fbt::cv_signal:return, +fbt::sleepq_broadcast:return +/self->trace_me_yo && self->stfu == stackdepth/ +{ + self->stfu = 0; +} + +/* This is where the actual tracing happens. */ +fbt::: +/self->trace_me_yo && ! self->stfu/ +{ + this->timestamp = walltimestamp; + this->msecs = (this->timestamp / 1000000) % 1000; + this->direction = probename == "entry" ? "-->" : "<--"; + printf("%d %d %Y.%.3d %.2d: %s %s %s:%s\n", + cpu, tid, this->timestamp, this->msecs, + stackdepth, indent[stackdepth], + this->direction, probefunc, probename); +} + +fbt::hpet_intr:entry, +fbt::lapic_handle_intr:entry, +fbt::termcn_cnputc:entry, +fbt::printf:entry, +fbt::cv_signal:entry, +fbt::sleepq_broadcast:entry +/self->trace_me_yo && !self->stfu/ +{ + this->timestamp = walltimestamp; + this->msecs = (this->timestamp / 1000000) % 1000; + printf("%d %d %Y.%.3d %.2d: %s [...]\n", + cpu, tid, this->timestamp, this->msecs, + stackdepth, indent[stackdepth]); + self->stfu = stackdepth; +} + +/* Done tracing */ +fbt::$$1:return +{ + self->trace_me_yo = 0; + exit(0); +} + +dtrace:::BEGIN +{ + /* perl -e 'for my $i (0..50) {printf(" indent[%s%d] = \"%s\";\n", $i < 10 ? " " : "", $i, " " x $i);}' */ + indent[ 0] = ""; + indent[ 1] = " "; + indent[ 2] = " "; + indent[ 3] = " "; + indent[ 4] = " "; + indent[ 5] = " "; + indent[ 6] = " "; + indent[ 7] = " "; + indent[ 8] = " "; + indent[ 9] = " "; + indent[10] = " "; + indent[11] = " "; + indent[12] = " "; + indent[13] = " "; + indent[14] = " "; + indent[15] = " "; + indent[16] = " "; + indent[17] = " "; + indent[18] = " "; + indent[19] = " "; + indent[20] = " "; + indent[21] = " "; + indent[22] = " "; + indent[23] = " "; + indent[24] = " "; + indent[25] = " "; + indent[26] = " "; + indent[27] = " "; + indent[28] = " "; + indent[29] = " "; + indent[30] = " "; + indent[31] = " "; + indent[32] = " "; + indent[33] = " "; + indent[34] = " "; + indent[35] = " "; + indent[36] = " "; + indent[37] = " "; + indent[38] = " "; + indent[39] = " "; + indent[40] = " "; + indent[41] = " "; + indent[42] = " "; + indent[43] = " "; + indent[44] = " "; + indent[45] = " "; + indent[46] = " "; + indent[47] = " "; + indent[48] = " "; + indent[49] = " "; + indent[50] = " "; +} -- 2.32.0 From 868b779b9008ebd2e03481a37775495e8500c414 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 26 Aug 2017 17:45:31 +0200 Subject: [PATCH 191/325] dmu_send.c: Call zfs_dbgmsg() when losing blkptrs or changing the block size Useful to help debugging ZFSonLinux bug #6224. Debug by running something like: dtrace -qn ' ::receive_object:entry { this->rwa=args[0]; this->drro=args[1]; } zfs-dbgmsg /this->rwa/ { printf("%Y: %s\n", walltimestamp, stringof(arg0)); print(*this->rwa); print(*this->rwa->os); } ::receive_object:return { this->rwa=0 }' while running the reproducer script. Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c index 1a7ba92fd9b3..fef98f4bdc63 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c @@ -2231,6 +2231,13 @@ receive_object(struct receive_writer_arg *rwa, struct drr_object *drro, if (drro->drr_blksz != doi.doi_data_block_size || nblkptr < doi.doi_nblkptr || dn_slots != doi.doi_dnodesize >> DNODE_SHIFT) { +#ifdef _KERNEL + zfs_dbgmsg("%s: calling dmu_free_long_range(). " + "drro->drr_blksz: %d, doi.doi_data_block_size: %d, " + "nblkptr: %d, doi.doi_nblkptr: %d, object: %d ", + __func__, drro->drr_blksz, doi.doi_data_block_size, + nblkptr, doi.doi_nblkptr, object); +#endif err = dmu_free_long_range(rwa->os, drro->drr_object, 0, DMU_OBJECT_END); if (err != 0) -- 2.32.0 From 57ddf56ae31367153d5e39cc851c01e206ba2bcd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 22 Aug 2017 10:01:08 +0200 Subject: [PATCH 192/325] sys/geom/eli: Ignore the passphrase set in the loader if it doesn't contain a single character Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 8411063478e2..d5a5d59570bd 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -104,9 +104,24 @@ fetch_loader_passphrase(void * dummy) KASSERT(dynamic_kenv, ("need dynamic kenv")); if ((env_passphrase = kern_getenv("kern.geom.eli.passphrase")) != NULL) { - /* Extract passphrase from the environment. */ - strlcpy(cached_passphrase, env_passphrase, - sizeof(cached_passphrase)); + if (env_passphrase[0] != '\0') { + /* Extract passphrase from the environment. */ + strlcpy(cached_passphrase, env_passphrase, + sizeof(cached_passphrase)); + G_ELI_DEBUG(0, "Using cached passphrase!"); + } else { + /* + * Passphrase is set in the environment but empty. + * This can happen if the user actually uses an empty + * passphrase (unlikely) and if the user pressed return + * to skip the passphrase prompt in the loader. + * + * In the second case we do not want to delay the boot + * by iterating over the empty and bogus passphrase and + * ask for another passphrase right away. + */ + G_ELI_DEBUG(0, "Ignoring cached passphrase as it seems to be empty!"); + } freeenv(env_passphrase); /* Wipe the passphrase from the environment. */ -- 2.32.0 From 0297e29908055437440f704b0ec2a3fae8c37e8c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 22 Aug 2017 11:18:26 +0200 Subject: [PATCH 193/325] sys/geom/eli: Show the environment passphrase if the visible_passphrase sysctl is set Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index d5a5d59570bd..54f10b2857c5 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -104,6 +104,8 @@ fetch_loader_passphrase(void * dummy) KASSERT(dynamic_kenv, ("need dynamic kenv")); if ((env_passphrase = kern_getenv("kern.geom.eli.passphrase")) != NULL) { + if (g_eli_visible_passphrase) + G_ELI_DEBUG(0, "Passphrase is %s", env_passphrase); if (env_passphrase[0] != '\0') { /* Extract passphrase from the environment. */ strlcpy(cached_passphrase, env_passphrase, -- 2.32.0 From cad45d381a1e0384837f91b10a813992b354e5ff Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 Sep 2017 13:38:24 +0200 Subject: [PATCH 194/325] zfs: Allow to toggle vfs.zfs.nopwrite_enabled at runtime While at it, declare the variable static as it's not referenced anywhere else. Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c index 45ac4b796fd2..03032c356d58 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c @@ -57,9 +57,9 @@ /* * Enable/disable nopwrite feature. */ -int zfs_nopwrite_enabled = 1; +static int zfs_nopwrite_enabled = 1; SYSCTL_DECL(_vfs_zfs); -SYSCTL_INT(_vfs_zfs, OID_AUTO, nopwrite_enabled, CTLFLAG_RDTUN, +SYSCTL_INT(_vfs_zfs, OID_AUTO, nopwrite_enabled, CTLFLAG_RWTUN, &zfs_nopwrite_enabled, 0, "Enable nopwrite feature"); /* -- 2.32.0 From ea29e05b74909c74a04d6f2ae976706cafe0322a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 11:56:04 +0200 Subject: [PATCH 195/325] acpi_ibm: Disable Bluetooth by default without looking how the hardware is inititialized Obtained from: ElectroBSD --- sys/dev/acpi_support/acpi_ibm.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sys/dev/acpi_support/acpi_ibm.c b/sys/dev/acpi_support/acpi_ibm.c index 31d0a0c04b0b..7b6fec7629ee 100644 --- a/sys/dev/acpi_support/acpi_ibm.c +++ b/sys/dev/acpi_support/acpi_ibm.c @@ -1023,6 +1023,10 @@ acpi_ibm_sysctl_init(struct acpi_ibm_softc *sc, int method) return (FALSE); case ACPI_IBM_METHOD_BLUETOOTH: + /* Disable bluetooth by default. */ + acpi_ibm_bluetooth_set(sc, 0); + + /* fallthrough */ case ACPI_IBM_METHOD_WLAN: if (ACPI_SUCCESS(acpi_GetInteger(sc->handle, IBM_NAME_WLAN_BT_GET, &dummy))) return (TRUE); -- 2.32.0 From e762b05c3d8306480d92c7de8c53002f9d54e6d9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 13:35:13 +0200 Subject: [PATCH 196/325] tools/build/options/makeman: Add silly hack to deal with options that have no description (XXX) If the option doesn't have a description file it's likely that the opposite option does have one. If it does, use its opossit and hope for the best. XXX: Does not confirm that the meaning of the description has actually been reverted by the sed calls. Obtained from: ElectroBSD --- tools/build/options/makeman | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/tools/build/options/makeman b/tools/build/options/makeman index 328b9faed881..ea544d542dd4 100755 --- a/tools/build/options/makeman +++ b/tools/build/options/makeman @@ -249,13 +249,26 @@ EOF show_options | while read opt targets ; do - if [ ! -f ${opt} ] ; then - echo "no description found for ${opt}, skipping" >&2 - continue + echo ".It Va ${opt}" + + if [ -f ${opt} ] ; then + sed -e'/\$FreeBSD.*\$/d' ${opt} + else + warning="no description found for ${opt}" + opposit_opt=WITH${opt#WITHOUT} + if [ -f ${opposit_opt} ] ; then + warning="${warning}. Using the opossite of ${opposit_opt}'s description" >&2 + sed -e'/\$FreeBSD.*\$/d' -E -e 's@(Set to) (build)@\1 not \2@' ${opposit_opt} + else + opposit_opt=WITHOUT${opt#WITH} + if [ -f ${opposit_opt} ] ; then + warning="${warning}. Using opossite of ${opposit_opt}'s description" >&2 + sed -e'/\$FreeBSD.*\$/d' -E -e 's@(Set to) not (build)@\1 \2@' ${opposit_opt} + fi + fi + echo "${warning}" >&2 fi - echo ".It Va ${opt}" - sed -e'/\$FreeBSD.*\$/d' ${opt} if [ -n "${targets}" ] ; then echo '.Pp' echo 'This is a default setting on' -- 2.32.0 From 6fdc9492dad4a847f97b2ca8eb79b3096557faaf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 13:40:58 +0200 Subject: [PATCH 197/325] Makefile: Explicitly reduce TARGETS to amd64 and i386 (XXX: workaround) Other targets currently aren't supported by ElectroBSD and somehow result in /usr/src/tools/build/options/makeman entering an endless loop. Reducing the targets doesn't impact the supported targets and works around the "makeman" issue for "now". Obtained from: ElectroBSD --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 21ce26f215c9..f0d650488fe7 100644 --- a/Makefile +++ b/Makefile @@ -492,7 +492,7 @@ worlds: .PHONY # .if make(universe) || make(universe_kernels) || make(tinderbox) || \ make(targets) || make(universe-toolchain) -TARGETS?=amd64 arm arm64 i386 mips powerpc riscv sparc64 +TARGETS?=amd64 i386 _UNIVERSE_TARGETS= ${TARGETS} TARGET_ARCHES_arm?= arm armv6 armv7 TARGET_ARCHES_arm64?= aarch64 -- 2.32.0 From 89ca8869b04cc08e74905f766ae7e602a994937b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 10 Sep 2017 16:41:47 +0200 Subject: [PATCH 198/325] zfs: Allow to set zfs_send_set_freerecords_bit at runtime Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c index fef98f4bdc63..47bb3f6428b5 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c @@ -74,12 +74,12 @@ int zfs_recv_queue_length = 16 * 1024 * 1024; int zfs_send_set_freerecords_bit = B_TRUE; #ifdef _KERNEL -TUNABLE_INT("vfs.zfs.send_set_freerecords_bit", &zfs_send_set_freerecords_bit); - SYSCTL_DECL(_vfs_zfs); SYSCTL_INT(_vfs_zfs, OID_AUTO, send_corrupt_data, CTLFLAG_RWTUN, &zfs_send_corrupt_data, 0, "Allow to send datasets with corrupt blocks. " "Blocks filled with 0x'zfs badd bloc' are send as replacement blocks."); +SYSCTL_INT(_vfs_zfs, OID_AUTO, send_set_freerecords_bit, CTLFLAG_RWTUN, + &zfs_send_set_freerecords_bit, 0, "Set the freerecords bits when sending"); #endif static char *dmu_recv_tag = "dmu_recv_tag"; -- 2.32.0 From 933ea05f7eb78c41556ec448ff3d02f8fc4f1fc5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 14 Sep 2017 15:26:22 +0200 Subject: [PATCH 199/325] ZFS: Shorten zone name for the 'metaslab allocation tracing facility' to work better with uma-summary Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c index 4dd6692d22f4..30de6a3e3a1c 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c @@ -3197,7 +3197,7 @@ metaslab_alloc_trace_init(void) { ASSERT(metaslab_alloc_trace_cache == NULL); metaslab_alloc_trace_cache = kmem_cache_create( - "metaslab_alloc_trace_cache", sizeof (metaslab_alloc_trace_t), + "metaslab_alloc_tcache", sizeof (metaslab_alloc_trace_t), 0, NULL, NULL, NULL, NULL, NULL, 0); metaslab_trace_ksp = kstat_create("zfs", 0, "metaslab_trace_stats", "misc", KSTAT_TYPE_NAMED, 1, KSTAT_FLAG_VIRTUAL); -- 2.32.0 From d8e2918d02120282dc6dcd4a1ff8493a61b2bd94 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 8 Oct 2017 12:20:03 +0200 Subject: [PATCH 200/325] sys/conf/newvers.sh: Fix detection of dirty git working trees Previously the .git directory was compared to the index which always resulted in the -dirty postfix getting added to the git revision: [fk@electrobsd /usr/src]$ git status On branch electrobsd nothing to commit, working tree clean [fk@electrobsd /usr/src]$ (cd /usr/obj/usr/src/sys/ELECTRO_BEER/ && sh -x /usr/src/sys/conf/newvers.sh 2>&1 | egrep 'VERINFO|work-tree') + /usr/local/bin/git '--git-dir=/usr/src/.git' '--work-tree=/usr/src/.git/objects/..' diff-index --name-only HEAD + VERINFO='ElectroBSD 11.1-STABLE #6 r324410+9c511ceaf6cf(electrobsd)-dirty: Sun Oct 8 12:29:36 CEST 2017' Obtained from: ElectroBSD --- sys/conf/newvers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index 782021850741..5bf092a9a8d5 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -223,7 +223,7 @@ if [ -n "$git_cmd" ] ; then if [ -n "$git_b" -a "$git_b" != "HEAD" ] ; then git="${git_b}-${git}" fi - if $git_cmd --work-tree=${VCSTOP} diff-index \ + if $git_cmd --work-tree=${VCSTOP%/objects}/.. diff-index \ --name-only HEAD | read dummy; then git="${git}-dirty" modified=true -- 2.32.0 From 258a9feffed994741eaf13d7e8e965361cdca0ef Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 11 Nov 2017 22:57:43 +0100 Subject: [PATCH 201/325] reproduce.sh: Use a LOG_TIMESTAMP_FORMAT variable. Squash? Obtained from: ElectroBSD --- reproduce.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/reproduce.sh b/reproduce.sh index 996212b07665..a0cc4bf7c0c7 100755 --- a/reproduce.sh +++ b/reproduce.sh @@ -58,12 +58,15 @@ RESUME_BUILD="${RESUME_BUILD-false}" # different number of cores. XZ_THREADS="${XZ_THREADS-2}" +# Used by announce_status() to create a timestamp prefix +LOG_TIMESTAMP_FORMAT='+%Y-%m-%d %H:%M:%S' + announce_status() { local msg \ timestamp msg="${*}" - timestamp=$(date "+%Y-%m-%d %H:%M") + timestamp=$(date "${LOG_TIMESTAMP_FORMAT}") echo "${timestamp}: ${msg}" } -- 2.32.0 From d3d07911e64b3a08e0abd8af6742707e95b9ef1c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 13 Nov 2017 21:41:37 +0100 Subject: [PATCH 202/325] i386: Don't try to build 3dfx as it gets removed by strip-freebsd.sh Obtained from: ElectroBSD --- sys/modules/Makefile | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index 4942c706e93c..b8aa8e1879d0 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -787,8 +787,10 @@ _vmm= vmm .if ${MACHINE_CPUARCH} == "i386" # XXX some of these can move to the general case when de-i386'ed # XXX some of these can move now, but are untested on other architectures. -_3dfx= 3dfx -_3dfx_linux= 3dfx_linux +# +# license unclear +#_3dfx= 3dfx +#_3dfx_linux= 3dfx_linux _aic= aic _apm= apm .if ${MK_SOURCELESS_UCODE} != "no" -- 2.32.0 From b54dd460c56f61bdbf93a5fa29132fd8eefbc9ff Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 14 Nov 2017 22:28:25 +0100 Subject: [PATCH 203/325] cam: Fix nullpointer dereference Unread portion of the kernel message buffer: ahcich1: Timeout on slot 27 port 0 ahcich1: is 00000000 cs 08000000 ss 00000000 rs 08000000 tfd 451 serr 00000000 cmd 000c5a17 Fatal trap 12: page fault while in kernel mode (aprobe0:ahcich1:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 00 40 00 00 00 00 08 00 cpuid = 2; apic id = 02 fault virtual address = 0x28 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff802a3981 stack pointer = 0x28:0xfffffe01ea3747e0 frame pointer = 0x28:0xfffffe01ea374810 (aprobe0:ahcich1:0:0:0): CAM status: Command timeout (aprobe0:ahcich1:0:0:0): Retrying command code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi4: clock (0)) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01ea3743c0 vpanic() at vpanic+0x186/frame 0xfffffe01ea374440 panic() at panic+0x43/frame 0xfffffe01ea3744a0 trap_fatal() at trap_fatal+0x34d/frame 0xfffffe01ea3744f0 trap_pfault() at trap_pfault+0x49/frame 0xfffffe01ea374550 trap() at trap+0x29a/frame 0xfffffe01ea374710 calltrap() at calltrap+0x8/frame 0xfffffe01ea374710 --- trap 0xc, rip = 0xffffffff802a3981, rsp = 0xfffffe01ea3747e0, rbp = 0xfffffe01ea374810 --- xpt_async() at xpt_async+0x331/frame 0xfffffe01ea374810 ahci_reset() at ahci_reset+0x1fe/frame 0xfffffe01ea374860 ahci_end_transaction() at ahci_end_transaction+0x8bd/frame 0xfffffe01ea3748c0 ahci_timeout() at ahci_timeout+0x334/frame 0xfffffe01ea374900 softclock_call_cc() at softclock_call_cc+0x13b/frame 0xfffffe01ea3749b0 softclock() at softclock+0xb9/frame 0xfffffe01ea3749e0 intr_event_execute_handlers() at intr_event_execute_handlers+0xec/frame 0xfffffe01ea374a20 ithread_loop() at ithread_loop+0xd6/frame 0xfffffe01ea374a70 fork_exit() at fork_exit+0x85/frame 0xfffffe01ea374ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01ea374ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 2h57m40s Dumping 1433 out of 8055 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% [...] __curthread () at ./machine/pcpu.h:222 222 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) where #0 __curthread () at ./machine/pcpu.h:222 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:298 #2 0xffffffff80579176 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #3 0xffffffff80579650 in vpanic (fmt=, ap=0xfffffe01ea374480) at /usr/src/sys/kern/kern_shutdown.c:759 #4 0xffffffff80579483 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #5 0xffffffff80855f6d in trap_fatal (frame=0xfffffe01ea374720, eva=40) at /usr/src/sys/amd64/amd64/trap.c:799 #6 0xffffffff80855fc9 in trap_pfault (frame=0xfffffe01ea374720, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:653 #7 0xffffffff8085581a in trap (frame=0xfffffe01ea374720) at /usr/src/sys/amd64/amd64/trap.c:420 #8 #9 0xffffffff802a3981 in xpt_async (async_code=1, path=, async_arg=0x0) at /usr/src/sys/cam/cam_xpt.c:4295 #10 0xffffffff8034d40e in ahci_reset (ch=0xfffffe0000d0c000) at /usr/src/sys/dev/ahci/ahci.c:2343 #11 0xffffffff8034ce2d in ahci_end_transaction (slot=, et=) at /usr/src/sys/dev/ahci/ahci.c:1957 #12 0xffffffff8034ea74 in ahci_process_timeout (ch=) at /usr/src/sys/dev/ahci/ahci.c:1671 #13 ahci_timeout (slot=0xfffffe0000d0ccb0) at /usr/src/sys/dev/ahci/ahci.c:1768 #14 0xffffffff805913ab in softclock_call_cc (c=, cc=0xffffffff80f08c80 , direct=) at /usr/src/sys/kern/kern_timeout.c:729 #15 0xffffffff80591909 in softclock (arg=0xffffffff80f08c80 ) at /usr/src/sys/kern/kern_timeout.c:867 #16 0xffffffff8053e89c in intr_event_execute_handlers (p=, ie=0xfffff8000326f500) at /usr/src/sys/kern/kern_intr.c:1262 #17 0xffffffff8053ef66 in ithread_execute_handlers (ie=, p=) at /usr/src/sys/kern/kern_intr.c:1275 #18 ithread_loop (arg=0xfffff800032487e0) at /usr/src/sys/kern/kern_intr.c:1356 #19 0xffffffff8053be85 in fork_exit (callout=0xffffffff8053ee90 , arg=0xfffff800032487e0, frame=0xfffffe01ea374ac0) at /usr/src/sys/kern/kern_fork.c:1044 #20 (kgdb) f 9 #9 0xffffffff802a3981 in xpt_async (async_code=1, path=, async_arg=0x0) at /usr/src/sys/cam/cam_xpt.c:4295 warning: Source file is more recent than executable. 4295 xpt_freeze_simq(path->bus->sim, 1); (kgdb) p path $1 = (kgdb) l - 4290 ccb->casync.async_arg_size = size; 4291 } 4292 if (path->device != NULL && path->device->lun_id != CAM_LUN_WILDCARD) 4293 xpt_freeze_devq(path, 1); 4294 else 4295 xpt_freeze_simq(path->bus->sim, 1); 4296 xpt_done(ccb); 4297 } 4298 4299 static void (kgdb) p path $2 = (kgdb) l - 4280 xpt_print(path, "Can't allocate argument to send %s\n", 4281 xpt_async_string(async_code)); 4282 xpt_free_path(ccb->ccb_h.path); 4283 xpt_free_ccb(ccb); 4284 return; 4285 } 4286 memcpy(ccb->casync.async_arg_ptr, async_arg, size); 4287 ccb->casync.async_arg_size = size; 4288 } else if (size < 0) { 4289 ccb->casync.async_arg_ptr = async_arg; (kgdb) l - 4270 size = xpt_async_size(async_code); 4271 CAM_DEBUG(ccb->ccb_h.path, CAM_DEBUG_TRACE, 4272 ("xpt_async: func %#x %s aync_code %d %s\n", 4273 ccb->ccb_h.func_code, 4274 xpt_action_name(ccb->ccb_h.func_code), 4275 async_code, 4276 xpt_async_string(async_code))); 4277 if (size > 0 && async_arg != NULL) { 4278 ccb->casync.async_arg_ptr = malloc(size, M_CAMXPT, M_NOWAIT); 4279 if (ccb->casync.async_arg_ptr == NULL) { (kgdb) f 10 #10 0xffffffff8034d40e in ahci_reset (ch=0xfffffe0000d0c000) at /usr/src/sys/dev/ahci/ahci.c:2343 2343 xpt_async(AC_BUS_RESET, ch->path, NULL); (kgdb) p ch->path $3 = (struct cam_path *) 0xfffff800032d3c80 (kgdb) p *ch->path $4 = {periph = 0x0, bus = 0x0, target = 0x0, device = 0x0} (kgdb) p *ch $5 = {dev = 0xfffff80003280b00, unit = 1, r_mem = 0xfffff80003474280, r_irq = 0xfffff80003474500, ih = 0x0, dma = {work_tag = 0xfffff80002fc8500, work_map = 0x0, work = 0xfffffe0233629000 "E", work_bus = 79859712, rfis_tag = 0xfffff80002fc8400, rfis_map = 0x0, rfis = 0xfffff80002fc8300 "\250\331\360YK\357\032q)\236\327\227\377^\332 -\341\260\300\356!J\237\065hr\304@a\\\036_`X", rfis_bus = 50103040, data_tag = 0xfffff80002fc8200}, sim = 0xfffff80003280400, path = 0xfffff800032d3c80, caps = 4281401189, caps2 = 4, chcaps = 786436, chscaps = 0, vendorid = 32902, deviceid = 7171, subvendorid = 6058, subdeviceid = 8655, quirks = 0, numslots = 32, pm_level = 0, devices = 1, pm_present = 0, fbs_enabled = 0, start = 0x0, hold = {0x0 }, slot = {{ch = 0xfffffe0000d0c000, slot = 0 '\000', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7ea00, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = { sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44295607766411, c_precision = 4026531562, c_arg = 0xfffffe0000d0c1b8, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 1 '\001', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e980, nsegs = 1}, timeout = {c_links = {le = {le_next = 0xfffffe0000d0ce10, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffffe0000d0ce10}, tqe = { tqe_next = 0xfffffe0000d0ce10, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45074001815689, c_precision = 4026531562, c_arg = 0xfffffe0000d0c220, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 2 '\002', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e900, nsegs = 0}, timeout = { c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45139272429188, c_precision = 4026531562, c_arg = 0xfffffe0000d0c288, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, { ch = 0xfffffe0000d0c000, slot = 3 '\003', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e880, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff800035bb3a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff800035bb3a0}, tqe = {tqe_next = 0xfffff800035bb3a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45203735588891, c_precision = 4026531562, c_arg = 0xfffffe0000d0c2f0, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 4 '\004', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e800, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = { tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44360878379910, c_precision = 4026531562, c_arg = 0xfffffe0000d0c358, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 5 '\005', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e780, nsegs = 0}, timeout = { c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44425332949679, c_precision = 4026531562, c_arg = 0xfffffe0000d0c3c0, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 6 '\006', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e700, nsegs = 1}, timeout = {c_links = {le = {le_next = 0xfffffe0000d0c318, le_prev = 0xffffffff80f08d98 }, sle = { sle_next = 0xfffffe0000d0c318}, tqe = {tqe_next = 0xfffffe0000d0c318, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45203735588891, c_precision = 4026531562, c_arg = 0xfffffe0000d0c428, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 7 '\a', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e680, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45268997612456, c_precision = 4026531562, c_arg = 0xfffffe0000d0c490, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 8 '\b', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e600, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45333439297324, c_precision = 4026531562, c_arg = 0xfffffe0000d0c4f8, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 9 '\t', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e580, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = { sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44490607858145, c_precision = 4026531562, c_arg = 0xfffffe0000d0c560, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 10 '\n', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e500, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff800035bb3a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff800035bb3a0}, tqe = { tqe_next = 0xfffff800035bb3a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44555071017848, c_precision = 4026531562, c_arg = 0xfffffe0000d0c5c8, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 11 '\v', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e480, nsegs = 1}, timeout = { c_links = {le = {le_next = 0xfffffe0000d0c520, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffffe0000d0c520}, tqe = {tqe_next = 0xfffffe0000d0c520, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45333443592291, c_precision = 4026531562, c_arg = 0xfffffe0000d0c630, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, { ch = 0xfffffe0000d0c000, slot = 12 '\f', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e400, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45398744270559, c_precision = 4026531562, c_arg = 0xfffffe0000d0c698, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 13 '\r', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e380, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff8001b2293a0}, tqe = { tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45463190250394, c_precision = 4026531562, c_arg = 0xfffffe0000d0c700, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 14 '\016', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e300, nsegs = 0}, timeout = { c_links = {le = {le_next = 0xfffff8001b2293a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff8001b2293a0}, tqe = {tqe_next = 0xfffff8001b2293a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44620333041413, c_precision = 4026531562, c_arg = 0xfffffe0000d0c768, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, { ch = 0xfffffe0000d0c000, slot = 15 '\017', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e280, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff800035bb3a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff800035bb3a0}, tqe = {tqe_next = 0xfffff800035bb3a0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 44684796201116, c_precision = 4026531562, c_arg = 0xfffffe0000d0c7d0, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 16 '\020', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e200, nsegs = 1}, timeout = {c_links = {le = {le_next = 0xfffffe0000d0c728, le_prev = 0xffffffff80f08d98 }, sle = { sle_next = 0xfffffe0000d0c728}, tqe = {tqe_next = 0xfffffe0000d0c728, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45463190250394, c_precision = 4026531562, c_arg = 0xfffffe0000d0c838, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 17 '\021', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = { data_map = 0xfffff80002f7e180, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45528469453827, c_precision = 4026531562, c_arg = 0xfffffe0000d0c8a0, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 18 '\022', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e100, nsegs = 0}, timeout = {c_links = {le = {le_next = 0x0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffffff80f08d98 }}, c_time = 45592924023596, c_precision = 4026531562, c_arg = 0xfffffe0000d0c908, c_func = 0xffffffff8034e740 , c_lock = 0xfffffe0000d0d300, c_flags = 2, c_iflags = 128, c_cpu = 0}}, {ch = 0xfffffe0000d0c000, slot = 19 '\023', state = AHCI_SLOT_EMPTY, ccb = 0x0, dma = {data_map = 0xfffff80002f7e080, nsegs = 0}, timeout = {c_links = {le = {le_next = 0xfffff800035bb3a0, le_prev = 0xffffffff80f08d98 }, sle = {sle_next = 0xfffff800035bb3a0}, tqe = { ---Type to continue, or q to quit---qq tqe_next Quit (kgdb) where #0 __curthread () at ./machine/pcpu.h:222 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:298 #2 0xffffffff80579176 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #3 0xffffffff80579650 in vpanic (fmt=, ap=0xfffffe01ea374480) at /usr/src/sys/kern/kern_shutdown.c:759 #4 0xffffffff80579483 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #5 0xffffffff80855f6d in trap_fatal (frame=0xfffffe01ea374720, eva=40) at /usr/src/sys/amd64/amd64/trap.c:799 #6 0xffffffff80855fc9 in trap_pfault (frame=0xfffffe01ea374720, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:653 #7 0xffffffff8085581a in trap (frame=0xfffffe01ea374720) at /usr/src/sys/amd64/amd64/trap.c:420 #8 #9 0xffffffff802a3981 in xpt_async (async_code=1, path=, async_arg=0x0) at /usr/src/sys/cam/cam_xpt.c:4295 #10 0xffffffff8034d40e in ahci_reset (ch=0xfffffe0000d0c000) at /usr/src/sys/dev/ahci/ahci.c:2343 #11 0xffffffff8034ce2d in ahci_end_transaction (slot=, et=) at /usr/src/sys/dev/ahci/ahci.c:1957 #12 0xffffffff8034ea74 in ahci_process_timeout (ch=) at /usr/src/sys/dev/ahci/ahci.c:1671 #13 ahci_timeout (slot=0xfffffe0000d0ccb0) at /usr/src/sys/dev/ahci/ahci.c:1768 #14 0xffffffff805913ab in softclock_call_cc (c=, cc=0xffffffff80f08c80 , direct=) at /usr/src/sys/kern/kern_timeout.c:729 #15 0xffffffff80591909 in softclock (arg=0xffffffff80f08c80 ) at /usr/src/sys/kern/kern_timeout.c:867 #16 0xffffffff8053e89c in intr_event_execute_handlers (p=, ie=0xfffff8000326f500) at /usr/src/sys/kern/kern_intr.c:1262 #17 0xffffffff8053ef66 in ithread_execute_handlers (ie=, p=) at /usr/src/sys/kern/kern_intr.c:1275 #18 ithread_loop (arg=0xfffff800032487e0) at /usr/src/sys/kern/kern_intr.c:1356 #19 0xffffffff8053be85 in fork_exit (callout=0xffffffff8053ee90 , arg=0xfffff800032487e0, frame=0xfffffe01ea374ac0) at /usr/src/sys/kern/kern_fork.c:1044 #20 Obtained from: ElectroBSD --- sys/cam/cam_xpt.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/sys/cam/cam_xpt.c b/sys/cam/cam_xpt.c index bd59dc1fc01f..36cb73141a55 100644 --- a/sys/cam/cam_xpt.c +++ b/sys/cam/cam_xpt.c @@ -4463,10 +4463,12 @@ xpt_async(u_int32_t async_code, struct cam_path *path, void *async_arg) ccb->casync.async_arg_ptr = async_arg; ccb->casync.async_arg_size = size; } - if (path->device != NULL && path->device->lun_id != CAM_LUN_WILDCARD) - xpt_freeze_devq(path, 1); - else - xpt_freeze_simq(path->bus->sim, 1); + if (path != NULL) { + if (path->device != NULL && path->device->lun_id != CAM_LUN_WILDCARD) + xpt_freeze_devq(path, 1); + else + xpt_freeze_simq(path->bus->sim, 1); + } xpt_action(ccb); } -- 2.32.0 From fb0418e6dc90d0d65ac437fa144c01e259e2f923 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 14 Nov 2017 22:23:50 +0100 Subject: [PATCH 204/325] umass: Prevent nullpointer dereference in xpt_freeze_devq() Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x28 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff802a2ed2 stack pointer = 0x28:0xfffffe023416b900 frame pointer = 0x28:0xfffffe023416b920 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 96046 (usbus1) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe023416b4e0 vpanic() at vpanic+0x186/frame 0xfffffe023416b560 panic() at panic+0x43/frame 0xfffffe023416b5c0 trap_fatal() at trap_fatal+0x322/frame 0xfffffe023416b610 trap_pfault() at trap_pfault+0x49/frame 0xfffffe023416b670 trap() at trap+0x298/frame 0xfffffe023416b830 calltrap() at calltrap+0x8/frame 0xfffffe023416b830 --- trap 0xc, rip = 0xffffffff802a2ed2, rsp = 0xfffffe023416b900, rbp = 0xfffffe023416b920 --- xpt_freeze_devq() at xpt_freeze_devq+0x12/frame 0xfffffe023416b920 umass_cam_cb() at umass_cam_cb+0x163/frame 0xfffffe023416b940 umass_t_bbb_status_callback() at umass_t_bbb_status_callback+0x527/frame 0xfffffe023416b9a0 usbd_callback_wrapper() at usbd_callback_wrapper+0x63f/frame 0xfffffe023416b9f0 usb_command_wrapper() at usb_command_wrapper+0x10c/frame 0xfffffe023416ba10 usb_callback_proc() at usb_callback_proc+0x76/frame 0xfffffe023416ba30 usb_process() at usb_process+0xd9/frame 0xfffffe023416ba70 fork_exit() at fork_exit+0x85/frame 0xfffffe023416bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe023416bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 15h52m32s Dumping 1548 out of 8055 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% [...] __curthread () at ./machine/pcpu.h:222 222 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) where #0 __curthread () at ./machine/pcpu.h:222 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:298 #2 0xffffffff80579b16 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #3 0xffffffff80579ff0 in vpanic (fmt=, ap=0xfffffe023416b5a0) at /usr/src/sys/kern/kern_shutdown.c:759 #4 0xffffffff80579e23 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #5 0xffffffff80853172 in trap_fatal (frame=0xfffffe023416b840, eva=40) at /usr/src/sys/amd64/amd64/trap.c:799 #6 0xffffffff808531c9 in trap_pfault (frame=0xfffffe023416b840, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:653 #7 0xffffffff80852a48 in trap (frame=0xfffffe023416b840) at /usr/src/sys/amd64/amd64/trap.c:420 #8 #9 xpt_freeze_devq (path=0xfffff80002fbac80, count=1) at /usr/src/sys/cam/cam_xpt.c:4340 #10 0xffffffff816649f3 in umass_cam_cb (sc=0xfffff800b8a28800, ccb=0xfffff80114f5d800, residue=0, status=3 '\003') at /usr/src/sys/dev/usb/storage/umass.c:2519 #11 0xffffffff816633c7 in umass_cancel_ccb (sc=) at /usr/src/sys/dev/usb/storage/umass.c:1151 #12 umass_tr_error (xfer=, error=) at /usr/src/sys/dev/usb/storage/umass.c:1167 #13 umass_t_bbb_status_callback (xfer=, error=) at /usr/src/sys/dev/usb/storage/umass.c:1568 #14 0xffffffff8163cf9f in usbd_callback_wrapper (pq=) at /usr/src/sys/dev/usb/usb_transfer.c:2408 #15 0xffffffff8163e06c in usb_command_wrapper (pq=0xfffffe002e97d060, xfer=) at /usr/src/sys/dev/usb/usb_transfer.c:3062 #16 0xffffffff8163d1d6 in usb_callback_proc (_pm=) at /usr/src/sys/dev/usb/usb_transfer.c:2269 #17 0xffffffff81638359 in usb_process (arg=0xfffffe0008596db0) at /usr/src/sys/dev/usb/usb_process.c:176 #18 0xffffffff8053cc05 in fork_exit (callout=0xffffffff81638280 , arg=0xfffffe0008596db0, frame=0xfffffe023416bac0) at /usr/src/sys/kern/kern_fork.c:1043 #19 (kgdb) f 9 #9 xpt_freeze_devq (path=0xfffff80002fbac80, count=1) at /usr/src/sys/cam/cam_xpt.c:4340 warning: Source file is more recent than executable. 4340 devq = dev->sim->devq; (kgdb) p dev->sim Cannot access memory at address 0x28 (kgdb) p dev $1 = (struct cam_ed *) 0x0 (kgdb) f 10 #10 0xffffffff816649f3 in umass_cam_cb (sc=0xfffff800b8a28800, ccb=0xfffff80114f5d800, residue=0, status=3 '\003') at /usr/src/sys/dev/usb/storage/umass.c:2519 2519 xpt_freeze_devq(ccb->ccb_h.path, 1); (kgdb) p ccb->ccb_h.path $2 = (struct cam_path *) 0xfffff80002fbac80 (kgdb) p *ccb->ccb_h.path $3 = {periph = 0xfffff8019bbcb100, bus = 0x0, target = 0x0, device = 0x0} (kgdb) p *ccb->ccb_h.path.dev There is no member named dev. (kgdb) p *ccb->ccb_h.path->device Cannot access memory at address 0x0 (kgdb) p *ccb->ccb_h.path.device Cannot access memory at address 0x0 (kgdb) p ccb->ccb_h.path.device $4 = (struct cam_ed *) 0x0 (kgdb) p ccb->ccb_h.path->device $5 = (struct cam_ed *) 0x0 Obtained from: ElectroBSD --- sys/dev/usb/storage/umass.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sys/dev/usb/storage/umass.c b/sys/dev/usb/storage/umass.c index 8a9657a5fb34..3b9d0420c769 100644 --- a/sys/dev/usb/storage/umass.c +++ b/sys/dev/usb/storage/umass.c @@ -2528,7 +2528,8 @@ umass_cam_cb(struct umass_softc *sc, union ccb *ccb, uint32_t residue, * recovered. We return an error to CAM and let CAM * retry the command if necessary. */ - xpt_freeze_devq(ccb->ccb_h.path, 1); + if (ccb->ccb_h.path != NULL) + xpt_freeze_devq(ccb->ccb_h.path, 1); ccb->ccb_h.status = CAM_REQ_CMP_ERR | CAM_DEV_QFRZN; xpt_done(ccb); break; -- 2.32.0 From 7659a25fb4fcfaebea4f25c1c7468d2c83cedd40 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 3 Jun 2017 10:32:51 +0200 Subject: [PATCH 205/325] Import reproduce-electrobsd.sh to 2018-01-21-cf9d75b Obtained from: ElectroBSD --- reproduce-electrobsd.sh | 151 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100755 reproduce-electrobsd.sh diff --git a/reproduce-electrobsd.sh b/reproduce-electrobsd.sh new file mode 100755 index 000000000000..908fc91f45bf --- /dev/null +++ b/reproduce-electrobsd.sh @@ -0,0 +1,151 @@ +#!/bin/sh + +########################################################################## +# Copyright (c) 2016-2017 Fabian Keil +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +########################################################################## + +# reproduce-electrobsd.sh +# +# This script builds ElectroBSD in a jail, updates the jail +# using the built binaries and builds ElectroBSD again. +# +# The second build should thus always result with the same +# binaries, even if different ElectroBSD versions were used +# for the first build, or (not yet tested) if the first build +# was done with FreeBSD binaries. +# +# The following steps are not yet automated: +# +# 1) Create an /etc/jail.conf with a section like this one: +# +# ElectroBSD-amd64 { +# host.hostname=ElectroBSD-amd64; +# persist; +# children.max=0; +# allow.mount; +# allow.mount.devfs; +# enforce_statfs=1; +# path=/usr/jails/ElectroBSD-amd64; +# exec.start="mount -t devfs devfs /dev"; +# exec.stop="umount /dev"; +# } +# +# 2) Create three new ZFS datasets for the build jail: +# +# /usr/jails/ElectroBSD-amd64 +# /usr/jails/ElectroBSD-amd64/usr +# /usr/jails/ElectroBSD-amd64/usr/src +# +# 3) Extract a base.txz in it. +# # tar xf /usr/electrobsd-dist/base.txz -C /usr/jails/ElectroBSD-amd64/ +# +# 4) Mount an UFS file system in "/usr/jails/ElectroBSD-amd64/usr/obj" +# and allow the BUILD_USER to write to it. This step is only required +# to get reproducible images, the tarballs should be reproducible +# when using ZFS as well. +# +# Example commands: +# +# # zfs create -V 20G -s -o volmode=geom dpool/ufs-obj-amd64 +# # newfs -t /dev/zvol/dpool/ufs-obj-amd64 +# # mount -o async,noatime /dev/zvol/dpool/ufs-obj-amd64 /usr/jails/ElectroBSD-amd64/usr/obj/ +# # chown fk:fk /usr/jails/ElectroBSD-amd64/usr/obj/ +# +# 5) Create a dedicated /usr/src dataset on the host and populate it +# with src.txz +# +# 6) Install sudo and zogftw on the host. + +HOST_SRC_DIR="/usr/src" +HOST_BUILDLOG_DIR="/var/log/buildlogs" + +BUILD_JAIL="${BUILD_JAIL-ElectroBSD-amd64}" +BUILD_JAIL_ROOT="/usr/jails/${BUILD_JAIL}" +BUILD_JAIL_SRC_DIR="${BUILD_JAIL_ROOT}/usr/src" +BUILD_USER="$(id -un)" +BUILD_NICE_VALUE=20 +BUILD_CORE_COUNT=2 +BUILD_TWICE="${BUILD_TWICE-true}" +# This is relative to the BUILD_JAIL_ROOT +BUILD_DISTFILE_DIR_PREFIX=/usr/obj/usr/src/ + +prepare_build_jail() { + local \ + src_dataset last_src_snapshot src_clone build_jail_dataset + + # Use vanilla zogftw configuration without potentionally + # existing fancy custom hooks that could slow use down. + export ZOGFTW_CONFIG_FILE='' + + zogftw snap /usr/src + + src_dataset=$(zogftw zcmd get_dataset_from_path "${HOST_SRC_DIR}") + if [ -z "${src_dataset}" ]; then + echo "Failed to get src dataset" + return 1 + fi + src_clone=$(zogftw zcmd get_dataset_from_path "${BUILD_JAIL_SRC_DIR}") + if [ -n "${src_clone}" ]; then + # The build jail already has a clone /usr/src, + # delete it as it may be stale + sudo zfs destroy "${src_clone}" || return 1 + else + build_jail_dataset=$(zogftw zcmd get_dataset_from_path "${BUILD_JAIL_ROOT}") + if [ -z "${build_jail_dataset}" ]; then + echo "Failed to get root jail dataset" + return 1 + fi + src_clone="${build_jail_dataset}/usr/src" + echo "Will create fresh clone in $src_clone" + fi + last_src_snapshot=$(zogftw zcmd get_last_snapshot "${src_dataset}") + if [ -z "${last_src_snapshot}" ]; then + echo "Failed to get src clone" + return 1 + fi + sudo zfs clone "${last_src_snapshot}" "${src_clone}" || return 1 +} + +reproduce() { + local \ + timestamp logfile build distfile_dir_prefix + + timestamp=$(date "+%Y-%m-%d_%H:%M") + logfile="${HOST_BUILDLOG_DIR}/reprolog-${timestamp}" + build="$(grep ^BUILD= /usr/src/reproduce.conf | cut -d = -f 2)" + distfile_dir_prefix="/usr/obj/usr/src/$build" + + script "${logfile}" \ + sudo nice -n "${BUILD_NICE_VALUE}" \ + sh -c "jail -c '${BUILD_JAIL}' && \ + jexec -u ${BUILD_USER} '${BUILD_JAIL}' \ + /usr/src/reproduce.sh -j${BUILD_CORE_COUNT} -d ${distfile_dir_prefix}-j1 -a && \ + jexec '${BUILD_JAIL}' make -C /usr/src installworld NO_FSCHG='yes' && \ + ${BUILD_TWICE} && jexec -u ${BUILD_USER} '${BUILD_JAIL}' \ + /usr/src/reproduce.sh -j${BUILD_CORE_COUNT} -d ${distfile_dir_prefix}-j2; + jail -r '${BUILD_JAIL}'" || return 1 + + # Compare checksums. Use a separate script call to make + # sure the checksums from the second run are flushed to + # the file + script -a "${logfile}" sh -c "grep ^SHA256 '${logfile}' | sort -k 4 | column -t" +} + +main() { + prepare_build_jail || return 1 + reproduce || return 1 +} + +main -- 2.32.0 From 90166038fbdd388b46f3dd5f7e6ac5dad45167a6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 12 May 2016 14:52:16 +0200 Subject: [PATCH 206/325] GELI: Use a dedicated uma zone for unauthenticated writes to onetime devices ... as they are likely to originate from the vm page daemon. Previously the system could deadlock because the vm daemon was waiting for pages to be written to disk, while GELI was waiting for the vm daemon to make room for the buffer GELI needed to actually write the pages: (kgdb) where #0 sched_switch (td=0xfffff800055bf9a0, newtd=0xfffff80002341000, flags=) at /usr/src/sys/kern/sched_ule.c:1969 #1 0xffffffff80962635 in mi_switch (flags=, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:455 #2 0xffffffff809aaa3a in sleepq_wait (wchan=0x0, pri=0) at /usr/src/sys/kern/subr_sleepqueue.c:637 #3 0xffffffff80962038 in _sleep (ident=, lock=, priority=, wmesg=0xffffffff80e826ee "vmwait", sbt=0, pr=, flags=) at /usr/src/sys/kern/kern_synch.c:229 #4 0xffffffff80c1ac6b in vm_wait () at /usr/src/sys/vm/vm_page.c:2705 #5 0xffffffff80c06a9f in kmem_back (object=0xffffffff8144d6f0, addr=18446741874805047296, size=69632, flags=) at /usr/src/sys/vm/vm_kern.c:356 #6 0xffffffff80c068d2 in kmem_malloc (vmem=0xffffffff813aa500, size=69632, flags=2) at /usr/src/sys/vm/vm_kern.c:316 #7 0xffffffff80bfd7d6 in uma_large_malloc (size=69632, wait=2) at /usr/src/sys/vm/uma_core.c:1106 #8 0xffffffff8092f614 in malloc (size=, mtp=0xffffffff81b4d520, flags=0) at /usr/src/sys/kern/kern_malloc.c:513 #9 0xffffffff81b4ab99 in g_eli_crypto_run (wr=0xfffff80002560040, bp=0xfffff80008a86d90) at /usr/src/sys/modules/geom/geom_eli/../../../geom/eli/g_eli_privacy.c:262 #10 0xffffffff81b3e860 in g_eli_worker (arg=0xfffff80002560040) at /usr/src/sys/modules/geom/geom_eli/../../../geom/eli/g_eli.c:565 #11 0xffffffff80910f5c in fork_exit (callout=0xffffffff81b3e0b0 , arg=0xfffff80002560040, frame=0xfffffe005005ec00) at /usr/src/sys/kern/kern_fork.c:1034 #12 0xffffffff80c33f0e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611 #13 0x0000000000000000 in ?? () (kgdb) p vm_cnt $16 = {v_swtch = 0, v_trap = 0, v_syscall = 0, v_intr = 0, v_soft = 0, v_vm_faults = 0, v_io_faults = 0, v_cow_faults = 0, v_cow_optim = 0, v_zfod = 0, v_ozfod = 0, v_swapin = 0, v_swapout = 0, v_swappgsin = 0, v_swappgsout = 0, v_vnodein = 0, v_vnodeout = 0, v_vnodepgsin = 0, v_vnodepgsout = 0, v_intrans = 0, v_reactivated = 0, v_pdwakeups = 22197, v_pdpages = 0, v_tcached = 0, v_dfree = 0, v_pfree = 0, v_tfree = 0, v_page_size = 4096, v_page_count = 247688, v_free_reserved = 372, v_free_target = 5320, v_free_min = 1609, v_free_count = 2, v_wire_count = 140735, v_active_count = 96194, v_inactive_target = 7980, v_inactive_count = 10756, v_cache_count = 0, v_pageout_free_min = 34, v_interrupt_free_min = 2, v_free_severe = 990, v_forks = 0, v_vforks = 0, v_rforks = 0, v_kthreads = 0, v_forkpages = 0, v_vforkpages = 0, v_rforkpages = 0, v_kthreadpages = 0, v_spare = 0xffffffff8144d5ac} A sysctl (kern.geom.eli.use_uma_for_all_writes) is added to optionally use the zone for unauthenticated GELI writes in general, without letting common writes cut into the reserve for onetime writes. This may reduce latency for larger writes and as we need to keep a couple of items in the zone anyway, the impact on the zone size is minor. Currently a single zone with a somewhat humongous item size sufficient for unauthenticated GELI writes up to MAXPHYS is being used. While this may look a bit wasteful, in practice we don't need a lot of items, so this seem tolerable for now. In case of writes that are too big to fit into an UMA zone item GELI falls back to using malloc again. The UMA zone item size is chosen based on kern.geom.eli.max_uma_bio_length. The pager never seems to use writes above MAXPHYS but ZFS will use writes up to recordsize. The best solution would probably be to only use the dedicated uma zone for common writes if the size is above 65356 bytes, the largest zone item size internally used by malloc. Currently the zone isn't used for reads as those are less time critical and usually are small enough for malloc() to succeed right away anyway. The zone currently isn't used for authenticated writes either, mainly because they aren't used by default for swapping and this would complicate the code without obvious gain. Example length distribution when reproducing ElectroBSD with -j4 and 1 GB of RAM: gpt/swap-ada1.eli BIO_WRITE value ------------- Distribution ------------- count < 4096 | 0 4096 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 4965848 8192 |@@@@@ 943980 12288 |@@ 362668 16384 |@ 161485 20480 |@ 120939 24576 | 87827 28672 | 57402 32768 | 40470 36864 | 42243 40960 | 28543 45056 | 20347 49152 | 15235 53248 | 13450 57344 | 9535 61440 | 9952 65536 |@ 179360 69632 | 0 gpt/swap-ada1.eli BIO_READ value ------------- Distribution ------------- count < 4096 | 0 4096 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 4645114 8192 | 0 12288 | 0 16384 | 3446 20480 | 0 Note that the GELI overhead is not accounted for here and only the results for the swap device are shown. Zone use: [fk@elektrobier3 ~]$ vmstat -z | egrep 'ITEM|eli' | column -t ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP g_eli: 172032, 0, 0, 14, 8077487, 0, 0 This includes writes to gpt/dpool-ada1.eli and gpt/dpool-ada1.eli. Discussion: While the zone served 8077487 memory requests total, 14 items were sufficient for this and therefore the zone only withheld 172032 * 14 bytes plus zone meta data from the rest of the system. PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209759 PR submission date: 2016-05-26 Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 58 ++++++++++++++++++++++++-- sys/geom/eli/g_eli.h | 3 ++ sys/geom/eli/g_eli_privacy.c | 81 +++++++++++++++++++++++++++++++++--- 3 files changed, 134 insertions(+), 8 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 54f10b2857c5..5aaaf44c92ad 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -86,6 +86,33 @@ u_int g_eli_batch = 0; SYSCTL_UINT(_kern_geom_eli, OID_AUTO, batch, CTLFLAG_RWTUN, &g_eli_batch, 0, "Use crypto operations batching"); +uma_zone_t g_eli_zone; +static u_int g_eli_uma_reserve = 1; +SYSCTL_UINT(_kern_geom_eli, OID_AUTO, uma_reserve, CTLFLAG_RDTUN, + &g_eli_uma_reserve, 0, "Items to pre-allocate in dedicated uma zone " + "and reserve for unauthenticated writes to onetime disks"); + +u_int g_eli_all_writes_use_uma = 0; +SYSCTL_UINT(_kern_geom_eli, OID_AUTO, use_uma_for_all_writes, CTLFLAG_RDTUN, + &g_eli_all_writes_use_uma, 0, "Use the dedicated uma zone for all " + "unauthenticated writes that fit, not just for writes to onetime " + "providers. May reduce write latency but also increases memory usage."); + +/* + * Items in the dedicated uma zone have a fixed size and need + * to be big enough for all bio lengths for which it is being used. + * + * g_eli_max_uma_bio_length is the largest amount of data geli can receive + * in a row and fit into an uma zone item, it is used to calculate + * g_eli_zone_item_size which additionally accounts for the encryption + * overhead, which depends on the number of sectors. + */ +u_int g_eli_max_uma_bio_length = MAXPHYS; +SYSCTL_UINT(_kern_geom_eli, OID_AUTO, max_uma_bio_length, CTLFLAG_RDTUN, + &g_eli_max_uma_bio_length, 0, + "Maximum bio length for which the uma zone is being used."); +u_int g_eli_zone_item_size; + /* * Passphrase cached during boot, in order to be more user-friendly if * there are multiple providers using the same passphrase. @@ -303,8 +330,8 @@ g_eli_write_done(struct bio *bp) pbp->bio_inbed++; if (pbp->bio_inbed < pbp->bio_children) return; - free(pbp->bio_driver2, M_ELI); - pbp->bio_driver2 = NULL; + sc = pbp->bio_to->geom->softc; + g_eli_free_driver2(pbp, sc->sc_flags); if (pbp->bio_error != 0) { G_ELI_LOGREQ(0, pbp, "%s() failed (error=%d)", __func__, pbp->bio_error); @@ -315,7 +342,6 @@ g_eli_write_done(struct bio *bp) /* * Write is finished, send it up. */ - sc = pbp->bio_to->geom->softc; g_io_deliver(pbp, pbp->bio_error); if (sc != NULL) atomic_subtract_int(&sc->sc_inflight, 1); @@ -1356,6 +1382,31 @@ static void g_eli_init(struct g_class *mp) { + /* + * Calculate uma zone item size based on the largest bio length + * we need to use it for. + * + * 512 bytes is the smallest sector size supported and results in the + * largest overhead. If larger sectors are being used, we'll just waste + * a bit more memory. + * + * We currently only use the zone for unauthenticated writes, + * otherwise the item size would have to be even bigger. + */ + g_eli_zone_item_size = (g_eli_max_uma_bio_length + \ + (g_eli_max_uma_bio_length / 512) * \ + (sizeof(struct cryptop) + sizeof(struct cryptodesc))); + + G_ELI_DEBUG(3, "Using uma zone item size %d for max bio length %d", + g_eli_zone_item_size, g_eli_max_uma_bio_length); + g_eli_zone = uma_zcreate("g_eli", g_eli_zone_item_size, NULL, NULL, + NULL, NULL, 0, UMA_ZONE_NOFREE); + if (0 < g_eli_uma_reserve) { + /* Increase the chances that items are available when needed. */ + uma_prealloc(g_eli_zone, g_eli_uma_reserve); + uma_zone_reserve(g_eli_zone, g_eli_uma_reserve); + } + g_eli_pre_sync = EVENTHANDLER_REGISTER(shutdown_pre_sync, g_eli_shutdown_pre_sync, mp, SHUTDOWN_PRI_FIRST); if (g_eli_pre_sync == NULL) @@ -1366,6 +1417,7 @@ static void g_eli_fini(struct g_class *mp) { + uma_zdestroy(g_eli_zone); if (g_eli_pre_sync != NULL) EVENTHANDLER_DEREGISTER(shutdown_pre_sync, g_eli_pre_sync); } diff --git a/sys/geom/eli/g_eli.h b/sys/geom/eli/g_eli.h index 5141081710a5..5cfb3935d3c4 100644 --- a/sys/geom/eli/g_eli.h +++ b/sys/geom/eli/g_eli.h @@ -147,6 +147,8 @@ #if (MAX_KEY_BYTES < G_ELI_DATAIVKEYLEN) #error "MAX_KEY_BYTES is less than G_ELI_DATAKEYLEN" #endif +extern u_int g_eli_max_uma_bio_length; +extern u_int g_eli_zone_item_size; extern int g_eli_debug; extern u_int g_eli_overwrites; @@ -722,5 +724,6 @@ void g_eli_key_init(struct g_eli_softc *sc); void g_eli_key_destroy(struct g_eli_softc *sc); uint8_t *g_eli_key_hold(struct g_eli_softc *sc, off_t offset, size_t blocksize); void g_eli_key_drop(struct g_eli_softc *sc, uint8_t *rawkey); +void g_eli_free_driver2(struct bio *bp, int sc_flags); #endif #endif /* !_G_ELI_H_ */ diff --git a/sys/geom/eli/g_eli_privacy.c b/sys/geom/eli/g_eli_privacy.c index 6dbda7f0ef03..99e60e0e677d 100644 --- a/sys/geom/eli/g_eli_privacy.c +++ b/sys/geom/eli/g_eli_privacy.c @@ -51,6 +51,9 @@ __FBSDID("$FreeBSD$"); #include #include +extern u_int g_eli_all_writes_use_uma; +extern uma_zone_t g_eli_zone; + /* * Code paths: * BIO_READ: @@ -61,6 +64,76 @@ __FBSDID("$FreeBSD$"); MALLOC_DECLARE(M_ELI); +static int +g_eli_use_uma_zone(const struct bio *bp, int sc_flags) +{ + + return (bp->bio_cmd == BIO_WRITE && + bp->bio_length <= g_eli_max_uma_bio_length && + ((sc_flags & G_ELI_FLAG_ONETIME) != 0 || g_eli_all_writes_use_uma) && + (sc_flags & G_ELI_FLAG_AUTH) == 0); +} + +static void * +g_eli_alloc_driver2(const struct bio *bp, size_t size, int sc_flags) +{ + void *p; + + /* + * While requests from the authentication code path should + * be handled correctly (by never using the uma zone) they + * currently aren't expected to occur at all. + */ + KASSERT((sc_flags & G_ELI_FLAG_AUTH) == 0, + ("g_eli_alloc_driver2() used for authenticated write")); + + if (g_eli_use_uma_zone(bp, sc_flags)) { + int uma_flags; + + KASSERT(size <= g_eli_zone_item_size, + ("Insufficient g_eli_zone_item_size %u < %u", + (unsigned)g_eli_zone_item_size, (unsigned)size)); + /* + * Writes to onetime providers are likely to originate + * from the page daemon, therefore we try to get the + * memory a bit harder for them to prevent vm deadlocks. + */ + if ((sc_flags & G_ELI_FLAG_ONETIME) != 0) + uma_flags = M_NOWAIT|M_USE_RESERVE; + else + uma_flags = M_WAITOK; + + G_ELI_DEBUG(3, "Using uma zone for size: %jd, bio_length: %jd", + (intmax_t)size, bp->bio_length); + while (NULL == (p = uma_zalloc(g_eli_zone, uma_flags))) { + /* Only reachable for onetime providers */ + pause("g_eli:uma", min(hz/1000, 1)); + } + } else { + G_ELI_DEBUG(3, "Using malloc for size: %jd, bio_length: %jd", + (intmax_t)size, bp->bio_length); + p = malloc(size, M_ELI, M_WAITOK); + } + + return (p); +} + +void +g_eli_free_driver2(struct bio *bp, int sc_flags) +{ + + if (g_eli_use_uma_zone(bp, sc_flags)) { + G_ELI_DEBUG(3, "Using uma_free for bio_length: %jd", + bp->bio_length); + uma_zfree(g_eli_zone, bp->bio_driver2); + } else { + G_ELI_DEBUG(3, "Using free for bio_length: %jd", + bp->bio_length); + free(bp->bio_driver2, M_ELI); + } + bp->bio_driver2 = NULL; +} + /* * The function is called after we read and decrypt data. * @@ -96,8 +169,7 @@ g_eli_crypto_read_done(struct cryptop *crp) */ if (bp->bio_inbed < bp->bio_children) return (0); - free(bp->bio_driver2, M_ELI); - bp->bio_driver2 = NULL; + g_eli_free_driver2(bp, sc->sc_flags); if (bp->bio_error != 0) { G_ELI_LOGREQ(0, bp, "Crypto READ request failed (error=%d).", bp->bio_error); @@ -155,8 +227,7 @@ g_eli_crypto_write_done(struct cryptop *crp) if (bp->bio_error != 0) { G_ELI_LOGREQ(0, bp, "Crypto WRITE request failed (error=%d).", bp->bio_error); - free(bp->bio_driver2, M_ELI); - bp->bio_driver2 = NULL; + g_eli_free_driver2(bp, sc->sc_flags); g_destroy_bio(cbp); g_io_deliver(bp, bp->bio_error); atomic_subtract_int(&sc->sc_inflight, 1); @@ -261,7 +332,7 @@ g_eli_crypto_run(struct g_eli_worker *wr, struct bio *bp) */ if (bp->bio_cmd == BIO_WRITE) size += bp->bio_length; - p = malloc(size, M_ELI, M_WAITOK); + p = g_eli_alloc_driver2(bp, size, sc->sc_flags); bp->bio_inbed = 0; bp->bio_children = nsec; -- 2.32.0 From a27a0206670c84d15ba0845e598e51e013d0266a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 15 Nov 2017 20:33:33 +0100 Subject: [PATCH 207/325] g_eli: Mark questionable cast Obtained from: ElectroBSD --- sys/geom/eli/g_eli_privacy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/geom/eli/g_eli_privacy.c b/sys/geom/eli/g_eli_privacy.c index 99e60e0e677d..6252c9a04919 100644 --- a/sys/geom/eli/g_eli_privacy.c +++ b/sys/geom/eli/g_eli_privacy.c @@ -92,7 +92,7 @@ g_eli_alloc_driver2(const struct bio *bp, size_t size, int sc_flags) KASSERT(size <= g_eli_zone_item_size, ("Insufficient g_eli_zone_item_size %u < %u", - (unsigned)g_eli_zone_item_size, (unsigned)size)); + (unsigned)/*XXX: why?*/g_eli_zone_item_size, (unsigned)size)); /* * Writes to onetime providers are likely to originate * from the page daemon, therefore we try to get the -- 2.32.0 From 5829afd2be86460135458c43e72935767a53fd64 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 22 Feb 2016 16:24:43 +0100 Subject: [PATCH 208/325] sys/modules: Skip mlx5 when building without OFED Fixes: ===> mlx (depend) machine -> /usr/src/sys/amd64/include x86 -> /usr/src/sys/x86/include awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/kern/device_if.m -h awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/kern/bus_if.m -h awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/dev/pci/pci_if.m -h rm -f .depend CC='cc' mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I/usr/src/sys -I/usr/obj/usr/src/sys/ELECTRO_BLOAT -D__printf__=__freebsd_kprintf__ -std=iso9899:1999 -include /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_global.h /usr/src/sys/modules/mlx/../../dev/mlx/mlx.c /usr/src/sys/modules/mlx/../../dev/mlx/mlx_pci.c /usr/src/sys/modules/mlx/../../dev/mlx/mlx_disk.c ===> mlx5 (depend) machine -> /usr/src/sys/amd64/include x86 -> /usr/src/sys/x86/include awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/kern/device_if.m -h awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/kern/bus_if.m -h awk -f /usr/src/sys/tools/vnode_if.awk /usr/src/sys/kern/vnode_if.src -p awk -f /usr/src/sys/tools/vnode_if.awk /usr/src/sys/kern/vnode_if.src -q awk -f /usr/src/sys/tools/vnode_if.awk /usr/src/sys/kern/vnode_if.src -h awk -f /usr/src/sys/tools/makeobjops.awk /usr/src/sys/dev/pci/pci_if.m -h ln -sf /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_inet.h opt_inet.h ln -sf /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_inet6.h opt_inet6.h ln -sf /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_random.h opt_random.h ln -sf /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_rss.h opt_rss.h rm -f .depend CC='cc' mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -I/usr/src/sys/modules/mlx5/../../ofed/include -I/usr/src/sys/modules/mlx5/../../compat/linuxkpi/common/include -DHAVE_KERNEL_OPTION_HEADERS -I. -I /usr/src/sys -I/usr/obj/usr/src/sys/ELECTRO_BLOAT -D__printf__=__freebsd_kprintf__ -std=iso9899:1999 -include /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_global.h /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core /mlx5_alloc.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cmd.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cq.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_eq.c /usr/src/sys /modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_eswitch_vacl.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_flow_table.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_fw.c /usr/src/sys/modules/ mlx5/../../dev/mlx5/mlx5_core/mlx5_health.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_mad.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_main.c /usr/src/sys/modules/mlx5/../../dev/mlx5/ mlx5_core/mlx5_mcg.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_mr.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_pagealloc.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_pd.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_port.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_qp.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_srq.c /usr/src/sys/modules/ml x5/../../dev/mlx5/mlx5_core/mlx5_transobj.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_uar.c /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_vport.c /usr/src/sys/modules/mlx5/../../dev/mlx5 /mlx5_core/mlx5_wq.c In file included from /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_alloc.c:33: In file included from /usr/src/sys/dev/mlx5/driver.h:41: /usr/src/sys/dev/mlx5/device.h:32:10: fatal error: 'rdma/ib_verbs.h' file not found ^ 1 error generated. In file included from /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cmd.c:38: In file included from /usr/src/sys/dev/mlx5/driver.h:41: /usr/src/sys/dev/mlx5/device.h:32:10: fatal error: 'rdma/ib_verbs.h' file not found ^ 1 error generated. In file included from /usr/src/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cq.c:31: In file included from /usr/src/sys/dev/mlx5/driver.h:41: /usr/src/sys/dev/mlx5/device.h:32:10: fatal error: 'rdma/ib_verbs.h' file not found ^ 1 error generated. [...] Obtained from: ElectroBSD --- sys/modules/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index b8aa8e1879d0..289eea8291fc 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -520,12 +520,14 @@ _ipoib= ipoib _iser= iser .endif _mlx4= mlx4 +.if ${MK_OFED} != "no" || defined(ALL_MODULES) _mlx5= mlx5 .if (${MK_INET_SUPPORT} != "no" && ${MK_INET6_SUPPORT} != "no") || \ defined(ALL_MODULES) _mlx4en= mlx4en _mlx5en= mlx5en .endif +.endif .if ${MK_OFED} != "no" || defined(ALL_MODULES) _mthca= mthca _mlx4ib= mlx4ib -- 2.32.0 From 7beb9137d55152ec868d019c836052e6d11529bf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 21 Feb 2018 10:41:24 +0100 Subject: [PATCH 209/325] Only build mlx4 if MK_OFED is set to yes. Squash cc -target x86_64-unknown-freebsd11.1 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -I/usr/src/sys/ofed/include -I/usr/src/sys/compat/ linuxkpi/common/include -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_global.h -I. -I/usr/src/sys -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/obj/usr/src/sys/ELEC TRO_BLOAT -MD -MF.depend.mlx4_catas.o -MTmlx4_catas.o -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -gdwarf-2 -Wall -Wredundant-decl s -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragma s -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno-error-shift-negative-value -Wno-error-address-of-packed-member -mno-aes -mno-av x -std=iso9899:1999 -Wno-cast-qual -Wno-pointer-arith -c /usr/src/sys/dev/mlx4/mlx4_core/mlx4_catas.c -o mlx4_catas.o ctfconvert -L VERSION -g mlx4_catas.o cc -target x86_64-unknown-freebsd11.1 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -I/usr/src/sys/ofed/include -I/usr/src/sys/compat/ linuxkpi/common/include -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/src/sys/ELECTRO_BLOAT/opt_global.h -I. -I/usr/src/sys -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/obj/usr/src/sys/ELEC TRO_BLOAT -MD -MF.depend.mlx4_cmd.o -MTmlx4_cmd.o -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -gdwarf-2 -Wall -Wredundant-decls -$ nested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -$ no-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno-error-shift-negative-value -Wno-error-address-of-packed-member -mno-aes -mno-avx $ std=iso9899:1999 -Wno-cast-qual -Wno-pointer-arith -c /usr/src/sys/dev/mlx4/mlx4_core/mlx4_cmd.c -o mlx4_cmd.o /usr/src/sys/dev/mlx4/mlx4_core/mlx4_cmd.c:45:10: fatal error: 'rdma/ib_smi.h' file not found #include ^~~~~~~~~~~~~~~ 1 error generated. *** Error code 1 Stop. make[4]: stopped in /usr/src/sys/modules/mlx4 *** Error code 1 Stop. make[3]: stopped in /usr/src/sys/modules *** Error code 1 Stop. make[2]: stopped in /usr/obj/usr/src/sys/ELECTRO_BLOAT *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src Obtained from: ElectroBSD --- sys/modules/Makefile | 2 -- 1 file changed, 2 deletions(-) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index 289eea8291fc..23b7ea0ef555 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -518,9 +518,7 @@ _ena= ena _ibcore= ibcore _ipoib= ipoib _iser= iser -.endif _mlx4= mlx4 -.if ${MK_OFED} != "no" || defined(ALL_MODULES) _mlx5= mlx5 .if (${MK_INET_SUPPORT} != "no" && ${MK_INET6_SUPPORT} != "no") || \ defined(ALL_MODULES) -- 2.32.0 From 9b224dfd2e6d34256247cc5c12e5154a5c10bd55 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 6 Apr 2018 17:51:04 +0200 Subject: [PATCH 210/325] Regenerate src.conf.5 (XXX: somewhat bogus) The content is somewhat bogus and contains duplicated content but keeping the upstream version makes even less sense as many defaults are different. File generated with: cd /usr/src/tools/build/options && sh makeman > ../../../share/man/man5/src.conf.5 Obtained from: ElectroBSD --- share/man/man5/src.conf.5 | 518 ++++++-------------------------------- 1 file changed, 80 insertions(+), 438 deletions(-) diff --git a/share/man/man5/src.conf.5 b/share/man/man5/src.conf.5 index 606e61a4e9c4..0060f022895a 100644 --- a/share/man/man5/src.conf.5 +++ b/share/man/man5/src.conf.5 @@ -1,6 +1,6 @@ .\" DO NOT EDIT-- this file is @generated by tools/build/options/makeman. .\" $FreeBSD$ -.Dd June 28, 2021 +.Dd October 31, 2021 .Dt SRC.CONF 5 .Os .Sh NAME @@ -101,8 +101,8 @@ Set to not build .Xr acpiconf 8 , .Xr acpidump 8 and related programs. -.It Va WITHOUT_AMD -Set to not build +.It Va WITH_AMD +Set to build .Xr amd 8 , and related programs. .It Va WITHOUT_APM @@ -118,8 +118,8 @@ checks. Set to not build .Xr at 1 and related utilities. -.It Va WITHOUT_ATM -Set to not build +.It Va WITH_ATM +Set to build programs and libraries related to ATM networking. .It Va WITHOUT_AUDIT Set to not build audit support into system programs. @@ -187,38 +187,15 @@ Do not build or install GNU .Xr objdump 1 as part of the normal system build. -.Pp -This is a default setting on -arm64/aarch64, riscv/riscv64 and riscv/riscv64sf. When set, it enforces these options: .Pp .Bl -item -compact .It .Va WITHOUT_GDB .El -.It Va WITH_BINUTILS -Set to build and install GNU -.Xr as 1 , -.Xr objdump 1 , -and for some CPU architectures -.Xr ld.bfd 1 -as part -of the normal system build. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. .It Va WITHOUT_BINUTILS_BOOTSTRAP Do not build GNU binutils as part of the bootstrap process. -.Pp -This is a default setting on -arm64/aarch64, riscv/riscv64 and riscv/riscv64sf. -.It Va WITH_BINUTILS_BOOTSTRAP -Build GNU binutils -as part of the bootstrap process. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. .It Va WITHOUT_BLACKLIST Set this if you do not want to build .Xr blacklistd 8 @@ -240,8 +217,8 @@ support, like .Xr ftpd 8 , and .Xr sshd 8 . -.It Va WITHOUT_BLUETOOTH -Set to not build Bluetooth related kernel modules, programs and libraries. +.It Va WITH_BLUETOOTH +Set to build Bluetooth related kernel modules, programs and libraries. .It Va WITHOUT_BOOT Set to not build the boot blocks and loader. .It Va WITHOUT_BOOTPARAMD @@ -250,8 +227,8 @@ Set to not build or install .It Va WITHOUT_BOOTPD Set to not build or install .Xr bootpd 8 . -.It Va WITHOUT_BSDINSTALL -Set to not build +.It Va WITH_BSDINSTALL +Set to build .Xr bsdinstall 8 , .Xr sade 8 , and related programs. @@ -265,6 +242,7 @@ and .Pa crtend.o . .It Va WITH_BSD_GREP Install BSD-licensed grep as '[ef]grep' instead of GNU grep. +.It Va WITHOUT_BSD_GREP_FASTMATCH .It Va WITHOUT_BSNMP Set to not build or install .Xr bsnmpd 1 @@ -339,8 +317,8 @@ CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' See .Xr ccache 1 for more configuration options. -.It Va WITHOUT_CCD -Set to not build +.It Va WITH_CCD +Set to build .Xr geom_ccd 4 and related utilities. .It Va WITHOUT_CDDL @@ -357,9 +335,6 @@ When set, it enforces these options: .El .It Va WITHOUT_CLANG Set to not build the Clang C/C++ compiler during the regular phase of the build. -.Pp -This is a default setting on -sparc64/sparc64. When set, it enforces these options: .Pp .Bl -item -compact @@ -372,58 +347,24 @@ When set, it enforces these options: .It .Va WITHOUT_LLVM_COV .El -.It Va WITH_CLANG -Set to build the Clang C/C++ compiler during the normal phase of the build. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_CLANG_BOOTSTRAP Set to not build the Clang C/C++ compiler during the bootstrap phase of the build. To be able to build the system, either gcc or clang bootstrap must be enabled unless an alternate compiler is provided via XCC. -.Pp -This is a default setting on -mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITH_CLANG_BOOTSTRAP -Set to build the Clang C/C++ compiler during the bootstrap phase of the build. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. -.It Va WITH_CLANG_EXTRAS -Set to build additional clang and llvm tools, such as bugpoint. +.It Va WITHOUT_CLANG_EXTRAS +Set to not build additional clang and llvm tools, such as bugpoint. .It Va WITH_CLANG_FORMAT Set to build clang-format. .It Va WITHOUT_CLANG_FULL Set to avoid building the ARCMigrate, Rewriter and StaticAnalyzer components of the Clang C/C++ compiler. -.Pp -This is a default setting on -sparc64/sparc64. -.It Va WITH_CLANG_FULL -Set to build the ARCMigrate, Rewriter and StaticAnalyzer components of the -Clang C/C++ compiler. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_CLANG_IS_CC Set to install the GCC compiler as .Pa /usr/bin/cc , .Pa /usr/bin/c++ and .Pa /usr/bin/cpp . -.Pp -This is a default setting on -mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITH_CLANG_IS_CC -Set to install the Clang C/C++ compiler as -.Pa /usr/bin/cc , -.Pa /usr/bin/c++ -and -.Pa /usr/bin/cpp . -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_CPP Set to not build .Xr cpp 1 . @@ -481,8 +422,6 @@ When set, it enforces these options: .Va WITHOUT_SVNLITE .It .Va WITHOUT_UNBOUND -.It -.Va WITHOUT_WIRELESS .El .Pp When set, these options are also in effect: @@ -497,24 +436,15 @@ is set explicitly) Set to compile with CTF (Compact C Type Format) data. CTF data encapsulates a reduced form of debugging information similar to DWARF and the venerable stabs and is required for DTrace. -.It Va WITHOUT_CTM -Set to not build +.It Va WITH_CTM +Set to build .Xr ctm 1 and related utilities. .It Va WITHOUT_CUSE Set to not build CUSE-related programs and libraries. -.It Va WITHOUT_CXGBETOOL -Set to not build -.Xr cxgbetool 8 -.Pp -This is a default setting on -arm/arm, arm/armv6, arm/armv7, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpcspe, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_CXGBETOOL Set to build .Xr cxgbetool 8 -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386, powerpc/powerpc64 and sparc64/sparc64. .It Va WITHOUT_CXX Set to not build .Xr c++ 1 @@ -673,17 +603,6 @@ Set not to build .Xr efivar 3 and .Xr efivar 8 . -.Pp -This is a default setting on -mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. -.It Va WITH_EFI -Set to build -.Xr efivar 3 -and -.Xr efivar 8 . -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64 and i386/i386. .It Va WITHOUT_ELFTOOLCHAIN_BOOTSTRAP Set to not build ELF Tool Chain tools (addr2line, nm, size, strings and strip) @@ -691,7 +610,7 @@ as part of the bootstrap process. .Bf -symbolic An alternate bootstrap tool chain must be provided. .Ef -.It Va WITHOUT_EXAMPLES +.It Va WITH_EXAMPLES Set to avoid installing examples to .Pa /usr/share/examples/ . .It Va WITH_EXTRA_TCP_STACKS @@ -714,11 +633,11 @@ Set to not build or install .Xr finger 1 and .Xr fingerd 8 . -.It Va WITHOUT_FLOPPY -Set to not build or install programs +.It Va WITH_FLOPPY +Set to build or install programs for operating floppy disk driver. -.It Va WITHOUT_FMTREE -Set to not build and install +.It Va WITH_FMTREE +Set to build and install .Pa /usr/sbin/fmtree . .It Va WITHOUT_FORMAT_EXTENSIONS Set to not enable @@ -731,98 +650,43 @@ Set to build bootloaders without Forth support. Set to build .Nm libc without floating-point support. -.It Va WITHOUT_FREEBSD_UPDATE -Set to not build +.It Va WITH_FREEBSD_UPDATE +Set to build .Xr freebsd-update 8 . -.It Va WITHOUT_FTP -Set to not build or install +.It Va WITH_FTP +Set to build or install .Xr ftp 1 and .Xr ftpd 8 . .It Va WITHOUT_GAMES Set to not build games. -.It Va WITHOUT_GCC -Set to not build and install gcc and g++ as part of the normal build process. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_GCC Set to build and install gcc and g++. -.Pp -This is a default setting on -mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITHOUT_GCC_BOOTSTRAP -Set to not build gcc and g++ as part of the bootstrap process. -You must enable either gcc or clang bootstrap to be able to build the system, -unless an alternative compiler is provided via -XCC. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_GCC_BOOTSTRAP Set to build gcc and g++ as part of the bootstrap process. -.Pp -This is a default setting on -mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITHOUT_GCOV -Set to not build the +.It Va WITH_GCOV +Build and install the GNU .Xr gcov 1 tool. .It Va WITHOUT_GDB Set to not build .Xr gdb 1 . -.Pp -This is a default setting on -arm64/aarch64, riscv/riscv64 and riscv/riscv64sf. -.It Va WITH_GDB -Set to build -.Xr gdb 1 . -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. .It Va WITHOUT_GDB_LIBEXEC Set to install .Xr gdb 1 into .Pa /usr/bin . -.Pp -This is a default setting on -sparc64/sparc64. -.It Va WITH_GDB_LIBEXEC -Set to install -.Xr gdb 1 -into -.Pa /usr/libexec . -This permits -.Xr gdb 1 -to be used as a fallback for -.Xr crashinfo 8 -if a newer version is not installed. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_GH_BC Set this option to install the enhanced .Xr bc 1 and .Xr dc 1 programs instead of the traditional FreeBSD versions. -.It Va WITHOUT_GNUCXX -Do not build the GNU C++ stack (g++, libstdc++). -This is the default on platforms where clang is the system compiler. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_GNUCXX Build the GNU C++ stack (g++, libstdc++). This is the default on platforms where gcc is the system compiler. -.Pp -This is a default setting on -mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. .It Va WITHOUT_GNU_DIFF Set to not build GNU -.Xr diff 1 -and .Xr diff3 1 . .It Va WITHOUT_GNU_GREP Set to not build GNU @@ -839,18 +703,9 @@ and dependent tests. Set to not build .Xr gpioctl 8 as part of the base system. -.It Va WITHOUT_GPL_DTC -Set to build the BSD licensed version of the device tree compiler rather -than the GPLed one from elinux.org. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_GPL_DTC Set to build the GPL'd version of the device tree compiler from elinux.org, instead of the BSD licensed one. -.Pp -This is a default setting on -mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. .It Va WITHOUT_GSSAPI Set to not build libgssapi. .It Va WITHOUT_HAST @@ -863,14 +718,6 @@ Set to build Hesiod support. Set to not build HTML docs. .It Va WITHOUT_HYPERV Set to not build or install HyperV utilities. -.Pp -This is a default setting on -arm/arm, arm/armv6, arm/armv7, arm64/aarch64, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. -.It Va WITH_HYPERV -Set to build or install HyperV utilities. -.Pp -This is a default setting on -amd64/amd64 and i386/i386. .It Va WITHOUT_ICONV Set to not build iconv as part of libc. .It Va WITHOUT_INCLUDES @@ -921,20 +768,20 @@ command. The user still must set the .Va DESTDIR variable to point to a directory where the user has write permissions. -.It Va WITHOUT_IPFILTER -Set to not build IP Filter package. +.It Va WITH_IPFILTER +Set to build IP Filter package. .It Va WITHOUT_IPFW Set to not build IPFW tools. -.It Va WITHOUT_IPSEC_SUPPORT -Set to not build the kernel with +.It Va WITH_IPSEC_SUPPORT +Set to build the kernel with .Xr ipsec 4 support. This option is needed for .Xr ipsec 4 and .Xr tcpmd5 4 . -.It Va WITHOUT_ISCSI -Set to not build +.It Va WITH_ISCSI +Set to build .Xr iscsid 8 and related utilities. .It Va WITHOUT_JAIL @@ -1040,91 +887,30 @@ Set to not build the library. .It Va WITHOUT_LLD Set to not build LLVM's lld linker. -.Pp -This is a default setting on -sparc64/sparc64. -.It Va WITH_LLD -Set to build LLVM's lld linker. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_LLDB Set to not build the LLDB debugger. -.Pp -This is a default setting on -arm/arm, arm/armv6, arm/armv7, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. -.It Va WITH_LLDB -Set to build the LLDB debugger. -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64 and i386/i386. .It Va WITHOUT_LLD_BOOTSTRAP Set to not build the LLD linker during the bootstrap phase of the build. To be able to build the system, either Binutils or LLD bootstrap must be enabled unless an alternate linker is provided via XLD. -.Pp -This is a default setting on -arm/arm, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITH_LLD_BOOTSTRAP -Set to build the LLD linker during the bootstrap phase of the build. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_LLD_IS_LD Set to use GNU binutils ld as the system linker, instead of LLVM's LLD. -.Pp -This is a default setting on -arm/arm, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITH_LLD_IS_LD -Set to use LLVM's LLD as the system linker, instead of GNU binutils ld. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_LLVM_ASSERTIONS Set to enable debugging assertions in LLVM. .It Va WITHOUT_LLVM_COV Set to not build the .Xr llvm-cov 1 tool. -.Pp -This is a default setting on -sparc64/sparc64. -.It Va WITH_LLVM_COV -Set to build the -.Xr llvm-cov 1 -tool. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_LLVM_CXXFILT Install LLVM's llvm-cxxfilt as c++filt, instead of ELF Tool Chain's cxxfilt. .It Va WITHOUT_LLVM_LIBUNWIND Set to use GCC's stack unwinder (instead of LLVM's libunwind). -.Pp -This is a default setting on -arm/arm, arm/armv6, arm/armv7, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITH_LLVM_LIBUNWIND -Set to use LLVM's libunwind stack unwinder (instead of GCC's unwinder). -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_LLVM_TARGET_AARCH64 Set to not build LLVM target support for AArch64. The .Va LLVM_TARGET_ALL option should be used rather than this in most cases. -.Pp -This is a default setting on -arm/arm. -.It Va WITH_LLVM_TARGET_AARCH64 -Set to build LLVM target support for AArch64. -The -.Va LLVM_TARGET_ALL -option should be used rather than this in most cases. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITHOUT_LLVM_TARGET_ALL Set to only build the required LLVM target support. This option is preferred to specific target support options. @@ -1171,81 +957,26 @@ Set to not build LLVM target support for MIPS. The .Va LLVM_TARGET_ALL option should be used rather than this in most cases. -.Pp -This is a default setting on -arm/arm. -.It Va WITH_LLVM_TARGET_MIPS -Set to build LLVM target support for MIPS. -The -.Va LLVM_TARGET_ALL -option should be used rather than this in most cases. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITHOUT_LLVM_TARGET_POWERPC Set to not build LLVM target support for PowerPC. The .Va LLVM_TARGET_ALL option should be used rather than this in most cases. -.Pp -This is a default setting on -arm/arm. -.It Va WITH_LLVM_TARGET_POWERPC -Set to build LLVM target support for PowerPC. -The -.Va LLVM_TARGET_ALL -option should be used rather than this in most cases. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITHOUT_LLVM_TARGET_RISCV Set to not build LLVM target support for RISC-V. The .Va LLVM_TARGET_ALL option should be used rather than this in most cases. -.Pp -This is a default setting on -arm/arm. -.It Va WITH_LLVM_TARGET_RISCV -Set to build LLVM target support for RISC-V. -The -.Va LLVM_TARGET_ALL -option should be used rather than this in most cases. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITHOUT_LLVM_TARGET_SPARC Set to not build LLVM target support for SPARC. The .Va LLVM_TARGET_ALL option should be used rather than this in most cases. -.Pp -This is a default setting on -arm/arm. -.It Va WITH_LLVM_TARGET_SPARC -Set to build LLVM target support for SPARC. -The -.Va LLVM_TARGET_ALL -option should be used rather than this in most cases. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITHOUT_LLVM_TARGET_X86 Set to not build LLVM target support for X86. The .Va LLVM_TARGET_ALL option should be used rather than this in most cases. -.Pp -This is a default setting on -arm/arm. -.It Va WITH_LLVM_TARGET_X86 -Set to build LLVM target support for X86. -The -.Va LLVM_TARGET_ALL -option should be used rather than this in most cases. -.Pp -This is a default setting on -amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITH_LOADER_EFI_SECUREBOOT Enable building .Xr loader 8 @@ -1259,44 +990,12 @@ Set to force the powerpc boot loader to launch the kernel in little endian mode. .It Va WITHOUT_LOADER_GELI Disable inclusion of GELI crypto support in the boot chain binaries. -.Pp -This is a default setting on -powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITH_LOADER_GELI -Set to build GELI bootloader support. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, riscv/riscv64 and riscv/riscv64sf. .It Va WITHOUT_LOADER_LUA Set to not build LUA bindings for the boot loader. -.Pp -This is a default setting on -powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITH_LOADER_LUA -Set to build LUA bindings for the boot loader. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, riscv/riscv64 and riscv/riscv64sf. -.It Va WITHOUT_LOADER_OFW -Disable building of openfirmware bootloader components. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_LOADER_OFW Set to build openfirmware bootloader components. -.Pp -This is a default setting on -powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe and sparc64/sparc64. -.It Va WITHOUT_LOADER_UBOOT -Disable building of ubldr. -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITH_LOADER_UBOOT Set to build ubldr. -.Pp -This is a default setting on -arm/arm, arm/armv6, arm/armv7, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64 and powerpc/powerpcspe. .It Va WITH_LOADER_VERIEXEC Enable building .Xr loader 8 @@ -1324,6 +1023,7 @@ The kernel has to be built with a module to parse the manifest. .Pp Depends on .Va WITH_LOADER_VERIEXEC . +.It Va WITH_LOADER_VERIEXEC_VECTX .It Va WITHOUT_LOADER_ZFS Set to not build ZFS file system boot loader support. .It Va WITHOUT_LOCALES @@ -1390,6 +1090,7 @@ is set explicitly) .It Va WITHOUT_MANCOMPRESS Set to not to install compressed man pages. Only the uncompressed versions will be installed. +.It Va WITHOUT_MANDOCDB .It Va WITHOUT_MAN_UTILS Set to not build utilities for manual pages, .Xr apropos 1 , @@ -1454,18 +1155,9 @@ This must be set in the environment, make command line, or .Pa /etc/src-env.conf , not .Pa /etc/src.conf . -.It Va WITHOUT_MLX5TOOL -Set to not build -.Xr mlx5tool 8 -.Pp -This is a default setting on -arm/arm, arm/armv6, arm/armv7, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpcspe, riscv/riscv64 and riscv/riscv64sf. .It Va WITH_MLX5TOOL Set to build .Xr mlx5tool 8 -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386, powerpc/powerpc64 and sparc64/sparc64. .It Va WITHOUT_MODULE_DRM Disable creation of old drm video modules. @@ -1473,34 +1165,25 @@ Disable creation of old drm video modules. Disable creation of old drm2 video modules. .It Va WITH_NAND Set to build the NAND Flash components. -.It Va WITHOUT_NDIS -Set to not build programs and libraries +.It Va WITH_NDIS +Set to build programs and libraries related to NDIS emulation support. .It Va WITHOUT_NETCAT Set to not build .Xr nc 1 utility. -.It Va WITHOUT_NETGRAPH -Set to not build applications to support +.It Va WITH_NETGRAPH +Set to build applications to support .Xr netgraph 4 . -When set, it enforces these options: -.Pp -.Bl -item -compact -.It -.Va WITHOUT_ATM -.It -.Va WITHOUT_BLUETOOTH -.El -.Pp When set, these options are also in effect: .Pp .Bl -inset -compact -.It Va WITHOUT_NETGRAPH_SUPPORT +.It Va WITH_NETGRAPH_SUPPORT (unless -.Va WITH_NETGRAPH_SUPPORT +.Va WITHOUT_NETGRAPH_SUPPORT is set explicitly) .El -.It Va WITHOUT_NETGRAPH_SUPPORT +.It Va WITH_NETGRAPH_SUPPORT Set to build libraries, programs, and kernel modules without netgraph support. .It Va WITHOUT_NIS Set to not build @@ -1535,18 +1218,10 @@ Set to not build and related programs. .It Va WITHOUT_NVME Set to not build nvme related tools and kernel modules. -.Pp -This is a default setting on -arm/arm, arm/armv6, arm/armv7, arm64/aarch64, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. -.It Va WITH_NVME -Set to build nvme related tools and kernel modules. -.Pp -This is a default setting on -amd64/amd64, i386/i386 and powerpc/powerpc64. .It Va WITH_OFED -Set to build the +Set to disable the build of the .Dq "OpenFabrics Enterprise Distribution" -Infiniband software stack. +Infiniband software stack, including kernel modules and userspace libraries. .It Va WITH_OFED_EXTRA Set to build the non-essential components of the .Dq "OpenFabrics Enterprise Distribution" @@ -1555,14 +1230,6 @@ Infiniband software stack, mostly examples. Enable building openldap support for kerberos. .It Va WITHOUT_OPENMP Set to not build LLVM's OpenMP runtime. -.Pp -This is a default setting on -arm/arm, arm/armv6, arm/armv7, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. -.It Va WITH_OPENMP -Set to build LLVM's OpenMP runtime. -.Pp -This is a default setting on -amd64/amd64, arm64/aarch64, i386/i386 and powerpc/powerpc64. .It Va WITHOUT_OPENSSH Set to not build OpenSSH. .It Va WITHOUT_OPENSSL @@ -1590,8 +1257,6 @@ When set, it enforces these options: .Va WITHOUT_SVNLITE .It .Va WITHOUT_UNBOUND -.It -.Va WITHOUT_WIRELESS .El .Pp When set, these options are also in effect: @@ -1620,8 +1285,8 @@ Set to build some programs without PAM support, particularly .Xr ftpd 8 and .Xr ppp 8 . -.It Va WITHOUT_PC_SYSINSTALL -Set to not build +.It Va WITH_PC_SYSINSTALL +Set to build .Xr pc-sysinstall 8 and related programs. .It Va WITHOUT_PF @@ -1639,32 +1304,21 @@ Position-Independent Executable (PIE). Set to not build .Xr pkg 7 bootstrap tool. -.It Va WITHOUT_PMC -Set to not build +.It Va WITH_PMC +Set to build .Xr pmccontrol 8 and related programs. -.It Va WITHOUT_PORTSNAP -Set to not build or install +.It Va WITH_PORTSNAP +Set to build or install .Xr portsnap 8 and related files. -.It Va WITHOUT_PPP -Set to not build +.It Va WITH_PPP +Set to build .Xr ppp 8 and related programs. .It Va WITHOUT_PROFILE Set to not build profiled libraries for use with .Xr gprof 8 . -.Pp -This is a default setting on -mips/mips64el, mips/mips64, mips/mips64elhf and mips/mips64hf. -.It Va WITH_PROFILE -Set to build profiled libraries for use with -.Xr gprof 8 . -This option is deprecated and is not present in -.Fx 14 . -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mipsn32, mips/mipselhf, mips/mipshf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITHOUT_QUOTAS Set to not build .Xr quota 1 @@ -1686,9 +1340,11 @@ and support in .Xr ifconfig 8 , by proxy. -.It Va WITHOUT_RBOOTD -Set to not build or install +.It Va WITH_RBOOTD +Set to build or install .Xr rbootd 8 . +.It Va WITH_RCMDS +.It Va WITHOUT_RCS .It Va WITHOUT_REPRODUCIBLE_BUILD Set to include build metadata (such as the build time, user, and host) in the kernel, boot loaders, and uname output. @@ -1699,16 +1355,17 @@ Set to not build .It Va WITH_RETPOLINE Set to build the base system with the retpoline speculative execution vulnerability mitigation for CVE-2017-5715. -.It Va WITHOUT_ROUTED -Set to not build +.It Va WITH_ROUTED +Set to build .Xr routed 8 utility. .It Va WITH_RPCBIND_WARMSTART_SUPPORT Set to build .Xr rpcbind 8 with warmstart support. -.It Va WITHOUT_SENDMAIL -Set to not build +.It Va WITH_SCTP_SUPPORT +.It Va WITH_SENDMAIL +Build and install .Xr sendmail 8 and related programs. .It Va WITHOUT_SERVICESDB @@ -1718,8 +1375,8 @@ Set to not install Set this to disable the installation of .Xr login 1 as a set-user-ID root program. -.It Va WITHOUT_SHAREDOCS -Set to not build the +.It Va WITH_SHAREDOCS +Set to build the .Bx 4.4 legacy docs. .It Va WITH_SHARED_TOOLCHAIN @@ -1731,30 +1388,14 @@ and necessary utilities like assembler, linker and library archive manager. .It Va WITH_SORT_THREADS Set to enable threads in .Xr sort 1 . -.It Va WITHOUT_SOURCELESS -Set to not build kernel modules that include sourceless code (either microcode or native code for host CPU). -When set, it enforces these options: -.Pp -.Bl -item -compact -.It -.Va WITHOUT_SOURCELESS_HOST -.It -.Va WITHOUT_SOURCELESS_UCODE -.El -.It Va WITHOUT_SOURCELESS_HOST -Set to not build kernel modules that include sourceless native code for host CPU. -.It Va WITHOUT_SOURCELESS_UCODE -Set to not build kernel modules that include sourceless microcode. +.It Va WITH_SOURCELESS +Set to build kernel modules that include sourceless code (either microcode or native code for host CPU). +.It Va WITH_SOURCELESS_HOST +Set to build kernel modules that include sourceless native code for host CPU. +.It Va WITH_SOURCELESS_UCODE +Set to build kernel modules that include sourceless microcode. .It Va WITHOUT_SSP Set to not build world with propolice stack smashing protection. -.Pp -This is a default setting on -mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf and mips/mips64hf. -.It Va WITH_SSP -Set to build world with propolice stack smashing protection. -.Pp -This is a default setting on -amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITH_STAGING Enable staging of files to a stage tree. This can be best thought of as auto-install to @@ -1790,8 +1431,8 @@ Set to install .Xr svnlite 1 as .Xr svn 1 . -.It Va WITHOUT_SVNLITE -Set to not build +.It Va WITH_SVNLITE +Set to build .Xr svnlite 1 and related programs. .It Va WITHOUT_SYMVER @@ -1800,6 +1441,7 @@ Set to disable symbol versioning when building shared libraries. Set to not build .Xr syscons 4 support files such as keyboard maps, fonts, and screen output maps. +.It Va WITH_SYSINSTALL .It Va WITH_SYSROOT Enable use of sysroot during build. Depends on @@ -1842,12 +1484,12 @@ Set to not build or install .Xr talk 1 and .Xr talkd 8 . -.It Va WITHOUT_TCP_WRAPPERS -Set to not build or install +.It Va WITH_TCP_WRAPPERS +Build and install .Xr tcpd 8 , and related utilities. -.It Va WITHOUT_TCSH -Set to not build and install +.It Va WITH_TCSH +Set to build and install .Pa /bin/csh (which is .Xr tcsh 1 ) . @@ -1894,13 +1536,13 @@ When set, it enforces these options: .It Va WITHOUT_TEXTPROC Set to not build programs used for text processing. -.It Va WITHOUT_TFTP -Set to not build or install +.It Va WITH_TFTP +Set to build or install .Xr tftp 1 and .Xr tftpd 8 . -.It Va WITHOUT_TIMED -Set to not build or install +.It Va WITH_TIMED +Set to build or install .Xr timed 8 . .It Va WITHOUT_TOOLCHAIN Set to not install header or -- 2.32.0 From 8f79f871150407662db672676ea386d1204e01d7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 16 Aug 2017 11:10:00 +0200 Subject: [PATCH 211/325] cam iosched: Fix trim statistics (XXX: may not be correct for nvme) When cam_iosched_bio_complete() gets called, all pending BIO_DELETEs are done. Previously only one of them was accounted for, as a result the "pending" counter got higher and higher and the "out" count was off. PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221952 PR submission date: 2017-08-31 Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 0b4d89468804..94a4497eebe9 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -1501,8 +1501,15 @@ cam_iosched_bio_complete(struct cam_iosched_softc *isc, struct bio *bp, } else if (bp->bio_cmd == BIO_DELETE) { if ((bp->bio_flags & BIO_ERROR) != 0) isc->trim_stats.errs++; - isc->trim_stats.out++; - isc->trim_stats.pending--; + /* + * A single request sent to the disk may trim multiple + * areas that belong to different BIO_DELETE requests. + * We only sent one trim request to the disk at the time, + * when it completes all pending BIO_DELETEs are done and + * have to be counted as such. + */ + isc->trim_stats.out += isc->trim_stats.pending; + isc->trim_stats.pending = 0; } else if (bp->bio_cmd != BIO_FLUSH) { if (iosched_debug) printf("Completing command with bio_cmd == %#x\n", bp->bio_cmd); -- 2.32.0 From 1fbeb62c5fe72a64da80ae0085588039238ba610 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 22 Jul 2017 20:12:40 +0200 Subject: [PATCH 212/325] cam_iosched: Add a sysctl for the iosched debug level The global iosched sysctls should probably get their own sysctl subtree ... Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 94a4497eebe9..845ea73a21c5 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -104,6 +104,9 @@ struct iop_stats; struct cam_iosched_softc; int iosched_debug = 0; +TUNABLE_INT("kern.cam.iosched_debug", &iosched_debug); +SYSCTL_INT(_kern_cam, OID_AUTO, iosched_debug, CTLFLAG_RW, + &iosched_debug, 1, "Dynamic I/O scheduler debug level"); typedef enum { none = 0, /* No limits */ -- 2.32.0 From 96a3dfa63e0d3ba69d26dba86ba2208573973bd3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 25 Jul 2017 13:06:48 +0200 Subject: [PATCH 213/325] # This is a combination of 2 commits. cam iosched: Optionally only enforce write limits if there are other requests pending ... or reads queued. This reduces write latency for many workloads without impacting read latency (too much). An example workload that profits a lot from this is watching a DVD rip from an USB disk while adding DVD rips using rsync. In this case the writes only have to be throttled when VLC is reading which only happens every couple of seconds. Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 845ea73a21c5..e58a79df73c3 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -283,6 +283,7 @@ struct cam_iosched_softc { int current_read_bias; /* Current read bias state */ int total_ticks; int load; /* EMA of 'load average' of disk / 2^16 */ + int ignore_write_limiter_if_no_ops_are_pending; struct bio_queue_head write_queue; struct iop_stats read_stats, write_stats, trim_stats; @@ -735,6 +736,10 @@ cam_iosched_has_io(struct cam_iosched_softc *isc) cam_iosched_limiter_caniop(&isc->write_stats, wbp) == 0; bool can_read = rbp != NULL && cam_iosched_limiter_caniop(&isc->read_stats, rbp) == 0; + if (isc->ignore_write_limiter_if_no_ops_are_pending && + !can_read && wbp != NULL && !can_write) { + can_write = 1; + } if (iosched_debug > 2) { printf("can write %d: pending_writes %d max_writes %d\n", can_write, isc->write_stats.pending, isc->write_stats.max); printf("can read %d: read_stats.pending %d max_reads %d\n", can_read, isc->read_stats.pending, isc->read_stats.max); @@ -1080,6 +1085,7 @@ cam_iosched_init(struct cam_iosched_softc **iscp, struct cam_periph *periph) (*iscp)->read_bias = 100; (*iscp)->current_read_bias = 100; (*iscp)->quanta = min(hz, 200); + (*iscp)->ignore_write_limiter_if_no_ops_are_pending = 0; cam_iosched_iop_stats_init(*iscp, &(*iscp)->read_stats); cam_iosched_iop_stats_init(*iscp, &(*iscp)->write_stats); cam_iosched_iop_stats_init(*iscp, &(*iscp)->trim_stats); @@ -1158,6 +1164,11 @@ void cam_iosched_sysctl_init(struct cam_iosched_softc *isc, &isc->read_bias, 100, "How biased towards read should we be independent of limits"); + SYSCTL_ADD_INT(ctx, n, + OID_AUTO, "ignore_write_limiter_if_no_ops_are_pending", CTLFLAG_RW, + &isc->ignore_write_limiter_if_no_ops_are_pending, 0, + "Only enforce write limits when there are ops pending"); + SYSCTL_ADD_PROC(ctx, n, OID_AUTO, "quanta", CTLTYPE_UINT | CTLFLAG_RW, &isc->quanta, 0, cam_iosched_quanta_sysctl, "I", @@ -1233,10 +1244,20 @@ cam_iosched_get_write(struct cam_iosched_softc *isc) * See if our current limiter allows this I/O. */ if (cam_iosched_limiter_iop(&isc->write_stats, bp) != 0) { - if (iosched_debug) - printf("Can't write because limiter says no.\n"); - isc->write_stats.state_flags |= IOP_RATE_LIMITED; - return NULL; + if (isc->ignore_write_limiter_if_no_ops_are_pending && + !bioq_first(&isc->bio_queue) && + isc->read_stats.pending == 0 && + isc->write_stats.pending == 0 && + isc->trim_stats.pending == 0) { + if (iosched_debug) + printf("Write limiter says no but no ops " + "are pending. Allowing write.\n"); + } else { + if (iosched_debug) + printf("Can't write because limiter says no.\n"); + isc->write_stats.state_flags |= IOP_RATE_LIMITED; + return NULL; + } } /* -- 2.32.0 From 9e3251c89dd0474824793fbdba05cbc978887b0d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 21 Jul 2017 16:11:21 +0200 Subject: [PATCH 214/325] cam iosched: Include the valid values in the cam_iosched_limiter_sysctl description Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index e58a79df73c3..b163abe912d8 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -987,7 +987,7 @@ cam_iosched_iop_stats_sysctl_init(struct cam_iosched_softc *isc, struct iop_stat SYSCTL_ADD_PROC(ctx, n, OID_AUTO, "limiter", CTLTYPE_STRING | CTLFLAG_RW, ios, 0, cam_iosched_limiter_sysctl, "A", - "Current limiting type."); + "Current limiting type. Valid values: 'none', 'queue_depth', 'iops' and 'bandwidth'"); SYSCTL_ADD_INT(ctx, n, OID_AUTO, "min", CTLFLAG_RW, &ios->min, 0, -- 2.32.0 From 840713250e54eacd868e90e08dbe9fe9c554068a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 2 Aug 2017 11:53:23 +0200 Subject: [PATCH 215/325] cam iosched: Include the accepted control type algorithms in the sysctl description ... so the user doesn't have to refer to the source code or the (non-existant) man page. Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index b163abe912d8..9c0cb47e9968 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -1033,7 +1033,7 @@ cam_iosched_cl_sysctl_init(struct cam_iosched_softc *isc) SYSCTL_ADD_PROC(ctx, n, OID_AUTO, "type", CTLTYPE_STRING | CTLFLAG_RW, clp, 0, cam_iosched_control_type_sysctl, "A", - "Control loop algorithm"); + "Control loop algorithm. Can be 'set_max' or 'read_latency'."); SYSCTL_ADD_PROC(ctx, n, OID_AUTO, "steer_interval", CTLTYPE_STRING | CTLFLAG_RW, &clp->steer_interval, 0, cam_iosched_sbintime_sysctl, "A", -- 2.32.0 From 4a397dcf4bd8a145564817fd8f459267d6b0eb8b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 8 Aug 2017 09:10:27 +0200 Subject: [PATCH 216/325] cam iosched: Only log the 'Steering write' message if writes are actually limited Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 9c0cb47e9968..df18d64bc42d 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -639,7 +639,8 @@ cam_iosched_cl_maybe_steer(struct control_loop *clp) clp->next_steer = now + clp->steer_interval; switch (clp->type) { case set_max: - if (isc->write_stats.current != isc->write_stats.max) + if (isc->write_stats.current != isc->write_stats.max && + isc->write_stats.limiter != none) { printf("Steering write from %d kBps to %d kBps\n", isc->write_stats.current, isc->write_stats.max); isc->read_stats.current = isc->read_stats.max; -- 2.32.0 From b5861a1b4bf523f70fcf572cb4e562f2a7a4489c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 3 Aug 2017 10:42:23 +0200 Subject: [PATCH 217/325] cam_iosched: Use the correct units when logging write limit changes ... for the 'queue_depth' or 'iops' limiter. Previously "kBps" were used unconditionally. Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index df18d64bc42d..6750c04f2358 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -119,6 +119,9 @@ typedef enum { static const char *cam_iosched_limiter_names[] = { "none", "queue_depth", "iops", "bandwidth" }; +static char *cam_iosched_limiter_units[] = + { "none", "queued ios", "iops", "kBps" }; + /* * Called to initialize the bits of the iop_stats structure relevant to the * limiter. Called just after the limiter is set. @@ -624,6 +627,14 @@ cam_iosched_cl_init(struct control_loop *clp, struct cam_iosched_softc *isc) clp->type = set_max; } +static const char * +cam_iosched_get_limiter_unit_string(enum io_limiter limiter) +{ + + KASSERT(none <= limiter && limiter < limiter_max, "Invalid limiter"); + return cam_iosched_limiter_units[limiter]; +} + static void cam_iosched_cl_maybe_steer(struct control_loop *clp) { @@ -641,8 +652,11 @@ cam_iosched_cl_maybe_steer(struct control_loop *clp) case set_max: if (isc->write_stats.current != isc->write_stats.max && isc->write_stats.limiter != none) { - printf("Steering write from %d kBps to %d kBps\n", - isc->write_stats.current, isc->write_stats.max); + const char *units; + units = cam_iosched_get_limiter_unit_string(isc->write_stats.limiter); + printf("Steering write from %d %s to %d %s\n", + isc->write_stats.current, units, isc->write_stats.max, units); + } isc->read_stats.current = isc->read_stats.max; isc->write_stats.current = isc->write_stats.max; isc->trim_stats.current = isc->trim_stats.max; -- 2.32.0 From d9a48b8feed9e54908efdbaf084d3b4acca15fdf Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 6 Aug 2017 19:01:53 +0200 Subject: [PATCH 218/325] cam iosched: Add sysctl to show the units used by the currently active limiter (XXX: or not) XXX: Doesn't work properly yet, sysctl always shows none. Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 6750c04f2358..e89b4674c156 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -211,6 +211,7 @@ struct iop_stats { * Information about the current rate limiters, if any */ io_limiter limiter; /* How are I/Os being limited */ + char *units; /* Current rate limiter unit (exported as sysctl) */ int min; /* Low range of limit */ int max; /* High range of limit */ int current; /* Current rate limiter */ @@ -627,8 +628,8 @@ cam_iosched_cl_init(struct control_loop *clp, struct cam_iosched_softc *isc) clp->type = set_max; } -static const char * -cam_iosched_get_limiter_unit_string(enum io_limiter limiter) +static char * +cam_iosched_get_limiter_unit_string(int limiter) { KASSERT(none <= limiter && limiter < limiter_max, "Invalid limiter"); @@ -652,10 +653,9 @@ cam_iosched_cl_maybe_steer(struct control_loop *clp) case set_max: if (isc->write_stats.current != isc->write_stats.max && isc->write_stats.limiter != none) { - const char *units; - units = cam_iosched_get_limiter_unit_string(isc->write_stats.limiter); printf("Steering write from %d %s to %d %s\n", - isc->write_stats.current, units, isc->write_stats.max, units); + isc->write_stats.current, isc->write_stats.units, + isc->write_stats.max, isc->write_stats.units); } isc->read_stats.current = isc->read_stats.max; isc->write_stats.current = isc->write_stats.max; @@ -798,6 +798,7 @@ cam_iosched_iop_stats_init(struct cam_iosched_softc *isc, struct iop_stats *ios) { ios->limiter = none; + ios->units = cam_iosched_get_limiter_unit_string(ios->limiter); ios->in = 0; ios->max = ios->current = 300000; ios->min = 1; @@ -840,9 +841,15 @@ cam_iosched_limiter_sysctl(SYSCTL_HANDLER_ARGS) if (strcmp(buf, cam_iosched_limiter_names[i]) != 0) continue; ios->limiter = i; + ios->units = cam_iosched_get_limiter_unit_string(i); error = cam_iosched_limiter_init(ios); if (error != 0) { + /* + * Continue with the previous limiter, + * ios->units remains valid. + */ ios->limiter = value; + ios->units = cam_iosched_get_limiter_unit_string(value); cam_periph_unlock(isc->periph); return error; } @@ -1003,6 +1010,10 @@ cam_iosched_iop_stats_sysctl_init(struct cam_iosched_softc *isc, struct iop_stat OID_AUTO, "limiter", CTLTYPE_STRING | CTLFLAG_RW, ios, 0, cam_iosched_limiter_sysctl, "A", "Current limiting type. Valid values: 'none', 'queue_depth', 'iops' and 'bandwidth'"); + SYSCTL_ADD_STRING(ctx, n, + OID_AUTO, "units", CTLFLAG_RD, + ios->units, 0, + "Units used by currently active limiter (read-only)"); SYSCTL_ADD_INT(ctx, n, OID_AUTO, "min", CTLFLAG_RW, &ios->min, 0, -- 2.32.0 From a69fc433d95152b538fa6c3c52906acc4d321766 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 25 Jul 2017 14:05:08 +0200 Subject: [PATCH 219/325] cam_iosched: Add sysctl to flush quickly Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index e89b4674c156..1517f1ae5477 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -108,6 +108,11 @@ TUNABLE_INT("kern.cam.iosched_debug", &iosched_debug); SYSCTL_INT(_kern_cam, OID_AUTO, iosched_debug, CTLFLAG_RW, &iosched_debug, 1, "Dynamic I/O scheduler debug level"); +int iosched_flush_quickly = 0; +TUNABLE_INT("kern.cam.iosched_flush_quickly", &iosched_flush_quickly); +SYSCTL_INT(_kern_cam, OID_AUTO, iosched_flush_quickly, CTLFLAG_RW, + &iosched_flush_quickly, 1, "Let flushes through if they are at the beginning of the queue"); + typedef enum { none = 0, /* No limits */ queue_depth, /* Limit how many ops we queue to SIM */ @@ -357,6 +362,9 @@ cam_iosched_limiter_caniop(struct iop_stats *ios, struct bio *bp) { int lim = ios->limiter; + if (iosched_flush_quickly && bp->bio_cmd == BIO_FLUSH) { + return 0; + } /* maybe this should be a kassert */ if (lim < none || lim >= limiter_max) return EINVAL; @@ -1255,15 +1263,17 @@ cam_iosched_get_write(struct cam_iosched_softc *isc) * setting. */ if (bioq_first(&isc->bio_queue) && isc->current_read_bias) { - if (iosched_debug) - printf( - "Reads present and current_read_bias is %d queued " - "writes %d queued reads %d\n", - isc->current_read_bias, isc->write_stats.queued, - isc->read_stats.queued); - isc->current_read_bias--; - /* We're not limiting writes, per se, just doing reads first */ - return NULL; + if (iosched_flush_quickly && bp->bio_cmd == BIO_FLUSH) { + if (iosched_debug) + printf("Reads pending but op is flush. Allowing it.\n"); + } else { + if (iosched_debug) + printf("Reads present and current_read_bias is %d queued writes %d queued reads %d\n", + isc->current_read_bias, isc->write_stats.queued, isc->read_stats.queued); + isc->current_read_bias--; + /* We're not limiting writes, per se, just doing reads first */ + return NULL; + } } /* -- 2.32.0 From 28d79e8b6b56c0e5724f53d5afd841d13e222090 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 5 Aug 2018 12:02:20 +0200 Subject: [PATCH 220/325] Unbreak the build Obtained from: ElectroBSD --- sys/cam/cam_iosched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 1517f1ae5477..8316d650249e 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -1291,7 +1291,7 @@ cam_iosched_get_write(struct cam_iosched_softc *isc) } else { if (iosched_debug) printf("Can't write because limiter says no.\n"); - isc->write_stats.state_flags |= IOP_RATE_LIMITED; + /* isc->write_stats.state_flags |= IOP_RATE_LIMITED; */ return NULL; } } -- 2.32.0 From fa14718206f1d265bdba93c61028a7c1b1239a58 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 11 Oct 2018 11:02:32 +0200 Subject: [PATCH 221/325] share/mk/src.opts.mk: Disable OFED by default on all platforms Obtained from: ElectroBSD --- share/mk/src.opts.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index fc34b3c072f5..b1732d245c1a 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -180,6 +180,7 @@ __DEFAULT_NO_OPTIONS = \ LOADER_FORCE_LE \ LOADER_VERIEXEC_PASS_MANIFEST \ NAND \ + OFED \ OFED_EXTRA \ OPENLDAP \ RPCBIND_WARMSTART_SUPPORT \ -- 2.32.0 From f8b808b3de50f1500aa7ed7c2c65c9b9a5317cec Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 11 Oct 2018 11:24:22 +0200 Subject: [PATCH 222/325] Allow to change the ostype at runtime Obtained from: ElectroBSD --- sys/kern/kern_mib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/kern_mib.c b/sys/kern/kern_mib.c index ef4891955279..f48d88eb7928 100644 --- a/sys/kern/kern_mib.c +++ b/sys/kern/kern_mib.c @@ -103,7 +103,7 @@ SYSCTL_INT(_kern, KERN_OSREV, osrevision, CTLFLAG_RD|CTLFLAG_CAPRD, SYSCTL_STRING(_kern, KERN_VERSION, version, CTLFLAG_RD|CTLFLAG_MPSAFE, version, 0, "Kernel version"); -SYSCTL_STRING(_kern, KERN_OSTYPE, ostype, CTLFLAG_RD|CTLFLAG_MPSAFE| +SYSCTL_STRING(_kern, KERN_OSTYPE, ostype, CTLFLAG_RW|CTLFLAG_MPSAFE| CTLFLAG_CAPRD, ostype, 0, "Operating system type"); SYSCTL_INT(_kern, KERN_MAXPROC, maxproc, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, -- 2.32.0 From b94a237c8f736109d4d17f5ec67af0ffcf31eca1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 26 Nov 2018 15:06:15 +0100 Subject: [PATCH 223/325] image-checksum.sh: Adjust to deal with slices instead of GPT partitions Obtained from: ElectroBSD --- release/scripts/image-checksum.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/release/scripts/image-checksum.sh b/release/scripts/image-checksum.sh index b8c1c94d0aaa..25cc7f58585a 100755 --- a/release/scripts/image-checksum.sh +++ b/release/scripts/image-checksum.sh @@ -45,7 +45,7 @@ # ########################################################################## -UFS_PARTITION=p2 +UFS_PARTITION=s2a EXPECTED_PARTITIONS=2 MOUNTPOINT=/mnt VERBOSE=0 @@ -96,7 +96,7 @@ partition_count_acceptable() { local md_unit="${1}" # Verify that there are exactly two partitions present - partitions=$(gpart show -r -p "md${md_unit}" | grep -c "md${md_unit}"p) + partitions=$(gpart show -r -p "md${md_unit}" | grep -c "md${md_unit}"s) if [ "${partitions}" != "${EXPECTED_PARTITIONS}" ]; then echo "Invalid number of partitions: ${partitions}" return 1; @@ -148,7 +148,7 @@ generate_partial_image_checksum() { mdconfig -d -u "${md_unit}" || return 1 - bootcode_checksum=$(dd if=/dev/md${md_unit}p1 2>/dev/null | sha256) + bootcode_checksum=$(dd if=/dev/md${md_unit}s1 2>/dev/null | sha256) verbose_log "Boot code checksum: ${bootcode_checksum}" mtree_checksum=$(sha256 -q "${spec_file}") verbose_log "mtree checksum: ${mtree_checksum}" -- 2.32.0 From 9b3f3cbf2eefc348664d1f09f07bbbcb607aadde Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 18 Jul 2019 18:27:05 +0200 Subject: [PATCH 224/325] dsl_dataset_hold_obj: Tolereate zap_lookup() failing with ENXIO Fixes a panic when all the vdevs dissappear while the pool is being imported. panic: solaris assert: zaperr == 0 (0x6 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c, line: 527 cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0234aae720 vpanic() at vpanic+0x17e/frame 0xfffffe0234aae780 panic() at panic+0x43/frame 0xfffffe0234aae7e0 assfail3() at assfail3+0x2c/frame 0xfffffe0234aae800 dsl_dataset_hold_obj() at dsl_dataset_hold_obj+0x380/frame 0xfffffe0234aae910 dmu_objset_find_dp_impl() at dmu_objset_find_dp_impl+0x285/frame 0xfffffe0234aae9a0 dmu_objset_find_dp_cb() at dmu_objset_find_dp_cb+0x25/frame 0xfffffe0234aae9c0 taskq_run() at taskq_run+0x10/frame 0xfffffe0234aae9e0 taskqueue_run_locked() at taskqueue_run_locked+0x154/frame 0xfffffe0234aaea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x98/frame 0xfffffe0234aaea70 fork_exit() at fork_exit+0x83/frame 0xfffffe0234aaeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0234aaeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Obtained from: ElectroBSD --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c index f226c0244004..a7c5d8f97a7c 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c @@ -523,7 +523,7 @@ dsl_dataset_hold_obj(dsl_pool_t *dp, uint64_t dsobj, void *tag, DS_FIELD_BOOKMARK_NAMES, sizeof (ds->ds_bookmarks), 1, &ds->ds_bookmarks); - if (zaperr != ENOENT) + if (zaperr != ENOENT && zaperr != ENXIO) VERIFY0(zaperr); } } else { -- 2.32.0 From 066c8821a449dbaeb6234c265f99b31226a58d80 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 12 Feb 2020 17:32:22 +0100 Subject: [PATCH 225/325] etc/shells: Remove /bin/csh and /bin/tcsh from the list of acceptable shells Obtained from: ElectroBSD --- lib/libc/gen/shells | 2 -- 1 file changed, 2 deletions(-) diff --git a/lib/libc/gen/shells b/lib/libc/gen/shells index fe1e0294a010..3797f57e85f9 100644 --- a/lib/libc/gen/shells +++ b/lib/libc/gen/shells @@ -5,5 +5,3 @@ # one of these shells. /bin/sh -/bin/csh -/bin/tcsh -- 2.32.0 From 5df3ca3d16612289dbaa1e3a553d04fc9d9def69 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 18 Feb 2020 19:26:06 +0100 Subject: [PATCH 226/325] strip-freebsd.sh: Add cp to the list of drivers to ditch (obfuscated code) Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 4b15c8e87f99..249417faa07f 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -102,6 +102,8 @@ get_tainted_sys_devs() { # use cases so figuring out the license problems is a low priority issue # because we can simply ditch the offending code. echo "tdfx" + # Intentionally obfuscated code in cpddk.c + echo "cp" } get_unused_contrib_dirs() { -- 2.32.0 From c883abbe41aaffe4010cfc05a41f9d7090280914 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 18 Feb 2020 19:30:51 +0100 Subject: [PATCH 227/325] strip-freebsd.sh: Add ce to the list of drivers to ditch (obfuscated code) Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 249417faa07f..c12d88416a03 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -102,7 +102,9 @@ get_tainted_sys_devs() { # use cases so figuring out the license problems is a low priority issue # because we can simply ditch the offending code. echo "tdfx" - # Intentionally obfuscated code in cpddk.c + # Intentionally obfuscated code and micro code in tau32-ddk.c + echo "ce" + # Intentionally obfuscated code and micro code in cpddk.c echo "cp" } -- 2.32.0 From 46a8687f2ce3dcd586dd767643eb57855f19e732 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 19 Feb 2020 13:02:01 +0100 Subject: [PATCH 228/325] release/amd64/make-memstick.sh: Use the freshly built mkimg Should prevent: Image `memstick.img.part' complete mkimg: invalid option -- a mkimg: error: unknown option usage: mkimg Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index c28521e9cbeb..7023f8b0a0fc 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -57,7 +57,7 @@ rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local rm "${2}.mtree" -mkimg -s mbr \ +dist/base/usr/bin/mkimg -s mbr \ -b ${1}/boot/mbr \ -p efi:=${1}/boot/boot1.efifat \ -p freebsd:-"mkimg -s bsd -b ${1}/boot/boot -p freebsd-ufs:=${2}.part" \ -- 2.32.0 From 18ad32a906eee7bf12268d1e552b896e4bb30141 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 1 May 2020 18:37:44 +0200 Subject: [PATCH 229/325] Detach mt and tcopy from the build Obtained from: ElectroBSD --- usr.bin/Makefile | 2 -- 1 file changed, 2 deletions(-) diff --git a/usr.bin/Makefile b/usr.bin/Makefile index cf506950632c..7006f985c4ad 100644 --- a/usr.bin/Makefile +++ b/usr.bin/Makefile @@ -97,7 +97,6 @@ SUBDIR= alias \ mklocale \ mktemp \ mkuzip \ - mt \ ncal \ netstat \ newgrp \ @@ -151,7 +150,6 @@ SUBDIR= alias \ tabs \ tail \ tar \ - tcopy \ tee \ time \ timeout \ -- 2.32.0 From 7a52c32bddb85a7fbffcd83d7a902c6f4d7fef76 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 2 May 2020 17:59:48 +0200 Subject: [PATCH 230/325] Detach gvinum from the build Obtained from: ElectroBSD --- sbin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/sbin/Makefile b/sbin/Makefile index 4b2908bffb04..c499db62865e 100644 --- a/sbin/Makefile +++ b/sbin/Makefile @@ -27,7 +27,6 @@ SUBDIR=adjkerntz \ geom \ ggate \ growfs \ - gvinum \ ifconfig \ init \ kldconfig \ -- 2.32.0 From aa9a188f783becc9b8e224f17ca100b1d0f8aa82 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 17 May 2020 15:20:43 +0200 Subject: [PATCH 231/325] release/rc.local: Adjust /tmp/reproduce-from-image.sh to deal with slices Obtained from: ElectroBSD --- release/rc.local | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/release/rc.local b/release/rc.local index a5977c3a8f84..877d3f4ac617 100755 --- a/release/rc.local +++ b/release/rc.local @@ -17,14 +17,14 @@ set -x -e # Fingers crossed that the first disk is the boot disk ... DISK=$(geom disk status -s | head -n 1 | cut -f 1 -w) JAILROOT="$(mktemp -d /tmp/ElectroBSD-jail-XXXXXX)" -SWAP_PARTITION_INDEX=3 -BUILD_PARTITION_INDEX=4 +SWAP_SLICE_INDEX=3 +BUILD_SLICE_INDEX=4 gpart recover "${DISK}" -gpart add -t freebsd-swap -s 1500M "${DISK}" -swapon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" -dumpon "/dev/${DISK}p${SWAP_PARTITION_INDEX}" -gpart add -t freebsd-ufs "${DISK}" +gpart add -t freebsd -s 1500M "${DISK}" +swapon "/dev/${DISK}s${SWAP_SLICE_INDEX}" +dumpon "/dev/${DISK}s${SWAP_SLICE_INDEX}" +gpart add -t freebsd "${DISK}" # gpart partition changes made in bhyve are currently # lost. The cause hasn't been diagnosed yet. # Explicitly include the partition layout in the @@ -32,8 +32,8 @@ gpart add -t freebsd-ufs "${DISK}" # afterwards. gpart show "${DISK}" # More convenient format for humans gpart backup "${DISK}" # Same data in format understood by "gpart restore" -newfs "/dev/${DISK}p${BUILD_PARTITION_INDEX}" -mount -o async,noatime "/dev/${DISK}p${BUILD_PARTITION_INDEX}" "${JAILROOT}" +newfs "/dev/${DISK}s${BUILD_SLICE_INDEX}" +mount -o async,noatime "/dev/${DISK}s${BUILD_SLICE_INDEX}" "${JAILROOT}" tar xf /usr/electrobsd-dist/base.txz -C "${JAILROOT}" tar xf /usr/electrobsd-dist/src.txz -C "${JAILROOT}" time jail -c path="${JAILROOT}" mount.devfs host.hostname=ElectroBSD command=sh /usr/src/reproduce.sh -- 2.32.0 From 93d46f4d3bfd0a176ce87839f9e0e86c7d0db165 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 30 May 2020 22:04:41 +0200 Subject: [PATCH 232/325] libdtrace: Fix comment typo Obtained from: ElectroBSD --- cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c index 589b30aa8654..afad27bef33c 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c @@ -1200,7 +1200,7 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *eprobesp) /* * We use this token as a relatively unique handle for this file on the * system in order to disambiguate potential conflicts between files of - * the same name which contain identially named local symbols. + * the same name which contain identically named local symbols. */ if ((objkey = hash_obj(obj, fd)) == (unsigned int)-1) return (dt_link_error(dtp, elf, fd, bufs, -- 2.32.0 From 4fdc15ef4ec0b0ed15c9d9b6bf740a20983519ee Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2020 08:46:52 +0200 Subject: [PATCH 233/325] share/man/man5/fstab.5: Fix typo Obtained from: ElectroBSD --- share/man/man5/fstab.5 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/man/man5/fstab.5 b/share/man/man5/fstab.5 index 13d91a1a6b78..d5cdf25dc858 100644 --- a/share/man/man5/fstab.5 +++ b/share/man/man5/fstab.5 @@ -426,7 +426,7 @@ resides in # Swap space on a block device. /dev/da0p1 none swap sw 0 0 # -# Swap space using a block device with GBDE/GELI encyption. +# Swap space using a block device with GBDE/GELI encryption. # aalgo, ealgo, keylen, sectorsize options are available # for .eli devices. /dev/da1p1.bde none swap sw 0 0 -- 2.32.0 From 5b8e7eb0be1d2a638296005a72a72279ba1ed8b0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 31 May 2020 15:37:36 +0200 Subject: [PATCH 234/325] Detach bsdconfig from the build Obtained from: ElectroBSD --- usr.sbin/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index 527002a6b04b..86a113e9147b 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -130,7 +130,6 @@ SUBDIR.${MK_OPENSSL}+= certctl .endif SUBDIR.${MK_CTM}+= ctm SUBDIR.${MK_CXGBETOOL}+= cxgbetool -SUBDIR.${MK_DIALOG}+= bsdconfig SUBDIR.${MK_EFI}+= efivar efidp efibootmgr .if ${MK_OPENSSL} != "no" SUBDIR.${MK_EFI}+= uefisign -- 2.32.0 From 25fbba2680b91338a66780dd88c9508c9b81bb91 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 2 Jun 2020 12:56:14 +0200 Subject: [PATCH 235/325] strip-freebsd.sh: Ditch ice files if available They are not yet available in stable/11 but one should always be prepared. Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index c12d88416a03..a420f0103f25 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -25,6 +25,18 @@ # likely to still contain lots of non-free parts that haven't # been discovered yet. +# ice was added in r361541/ba15208e9900 and is not (yet) +# available in stable/11. It relies on a proprietary +# "DDP package file" (sys/contrib/dev/ice/ice-1.3.9.0.pkg) +get_dev_ice_files() { + if [ -d sys/contrib/dev/ice ]; then + find sys/contrib/dev/ice + fi + if [ -d sys/dev/ice ]; then + find sys/dev/ice || true + fi +} + get_snd_csa_files() { find sys/dev/sound/pci -name "csa*" } @@ -56,6 +68,7 @@ get_usb_firmware_files() { # by non-free dependencies get_unfree_files() { find sys/ -name "*.uu" + get_dev_ice_files get_snd_csa_files get_snd_ds1_files get_snd_maestro3_files -- 2.32.0 From c5fc88cea5a985ac752b2ab62271d1420a39fa36 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 2 Jun 2020 18:48:53 +0200 Subject: [PATCH 236/325] printf: Let printf_doformat() use malloc() to work around FreeBSD bug 246630 Obtained from: ElectroBSD --- usr.bin/printf/printf.c | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/usr.bin/printf/printf.c b/usr.bin/printf/printf.c index 94667f381a60..53b9c5607e06 100644 --- a/usr.bin/printf/printf.c +++ b/usr.bin/printf/printf.c @@ -214,11 +214,15 @@ printf_doformat(char *fmt, int *rval) static const char skip1[] = "#'-+ 0"; int fieldwidth, haveprec, havewidth, mod_ldbl, precision; char convch, nextch; - char start[strlen(fmt) + 1]; + char *start; char **fargv; char *dptr; int l; + if ((start = malloc(strlen(fmt) + 1)) == NULL) { + warnx("%s", strerror(ENOMEM)); + return (NULL); + } dptr = start; *dptr++ = '%'; *dptr = 0; @@ -258,6 +262,7 @@ printf_doformat(char *fmt, int *rval) int idx = atoi(fmt); if (fargv == NULL) { warnx("incomplete use of n$"); + free(start); return (NULL); } if (idx <= myargc) { @@ -268,11 +273,14 @@ printf_doformat(char *fmt, int *rval) fmt += l + 1; } else if (fargv != NULL) { warnx("incomplete use of n$"); + free(start); return (NULL); } - if (getint(&fieldwidth)) + if (getint(&fieldwidth)) { + free(start); return (NULL); + } if (gargv > maxargv) maxargv = gargv; havewidth = 1; @@ -302,6 +310,7 @@ printf_doformat(char *fmt, int *rval) int idx = atoi(fmt); if (fargv == NULL) { warnx("incomplete use of n$"); + free(start); return (NULL); } if (idx <= myargc) { @@ -312,11 +321,14 @@ printf_doformat(char *fmt, int *rval) fmt += l + 1; } else if (fargv != NULL) { warnx("incomplete use of n$"); + free(start); return (NULL); } - if (getint(&precision)) + if (getint(&precision)) { + free(start); return (NULL); + } if (gargv > maxargv) maxargv = gargv; haveprec = 1; @@ -335,6 +347,7 @@ printf_doformat(char *fmt, int *rval) haveprec = 0; if (!*fmt) { warnx("missing format character"); + free(start); return (NULL); } *dptr++ = *fmt; @@ -355,6 +368,7 @@ printf_doformat(char *fmt, int *rval) fmt++; if (!strchr("aAeEfFgG", *fmt)) { warnx("bad modifier L for %%%c", *fmt); + free(start); return (NULL); } } else { @@ -380,6 +394,7 @@ printf_doformat(char *fmt, int *rval) start[strlen(start) - 1] = 's'; if ((p = strdup(getstr())) == NULL) { warnx("%s", strerror(ENOMEM)); + free(start); return (NULL); } getout = escape(p, 0, &len); @@ -387,8 +402,10 @@ printf_doformat(char *fmt, int *rval) /* Restore format for next loop. */ free(p); - if (getout) + if (getout) { + free(start); return (end_fmt); + } break; } case 'c': { @@ -413,8 +430,10 @@ printf_doformat(char *fmt, int *rval) int signedconv; signedconv = (convch == 'd' || convch == 'i'); - if ((f = mknum(start, convch)) == NULL) + if ((f = mknum(start, convch)) == NULL) { + free(start); return (NULL); + } if (getnum(&val, &uval, signedconv)) *rval = 1; if (signedconv) @@ -439,9 +458,11 @@ printf_doformat(char *fmt, int *rval) } default: warnx("illegal format character %c", convch); + free(start); return (NULL); } *fmt = nextch; + free(start); /* return the gargv to the next element */ return (fmt); } -- 2.32.0 From be13ccc67f2e566409afa30095175d55999921ee Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 3 Jun 2020 19:07:04 +0200 Subject: [PATCH 237/325] Revert "printf: Let printf_doformat() use malloc() to work around FreeBSD bug 246630" This reverts commit 476964e0034f0e69e9f7b99f81de7b8a2c30fba1. Obtained from: ElectroBSD --- usr.bin/printf/printf.c | 31 +++++-------------------------- 1 file changed, 5 insertions(+), 26 deletions(-) diff --git a/usr.bin/printf/printf.c b/usr.bin/printf/printf.c index 53b9c5607e06..94667f381a60 100644 --- a/usr.bin/printf/printf.c +++ b/usr.bin/printf/printf.c @@ -214,15 +214,11 @@ printf_doformat(char *fmt, int *rval) static const char skip1[] = "#'-+ 0"; int fieldwidth, haveprec, havewidth, mod_ldbl, precision; char convch, nextch; - char *start; + char start[strlen(fmt) + 1]; char **fargv; char *dptr; int l; - if ((start = malloc(strlen(fmt) + 1)) == NULL) { - warnx("%s", strerror(ENOMEM)); - return (NULL); - } dptr = start; *dptr++ = '%'; *dptr = 0; @@ -262,7 +258,6 @@ printf_doformat(char *fmt, int *rval) int idx = atoi(fmt); if (fargv == NULL) { warnx("incomplete use of n$"); - free(start); return (NULL); } if (idx <= myargc) { @@ -273,14 +268,11 @@ printf_doformat(char *fmt, int *rval) fmt += l + 1; } else if (fargv != NULL) { warnx("incomplete use of n$"); - free(start); return (NULL); } - if (getint(&fieldwidth)) { - free(start); + if (getint(&fieldwidth)) return (NULL); - } if (gargv > maxargv) maxargv = gargv; havewidth = 1; @@ -310,7 +302,6 @@ printf_doformat(char *fmt, int *rval) int idx = atoi(fmt); if (fargv == NULL) { warnx("incomplete use of n$"); - free(start); return (NULL); } if (idx <= myargc) { @@ -321,14 +312,11 @@ printf_doformat(char *fmt, int *rval) fmt += l + 1; } else if (fargv != NULL) { warnx("incomplete use of n$"); - free(start); return (NULL); } - if (getint(&precision)) { - free(start); + if (getint(&precision)) return (NULL); - } if (gargv > maxargv) maxargv = gargv; haveprec = 1; @@ -347,7 +335,6 @@ printf_doformat(char *fmt, int *rval) haveprec = 0; if (!*fmt) { warnx("missing format character"); - free(start); return (NULL); } *dptr++ = *fmt; @@ -368,7 +355,6 @@ printf_doformat(char *fmt, int *rval) fmt++; if (!strchr("aAeEfFgG", *fmt)) { warnx("bad modifier L for %%%c", *fmt); - free(start); return (NULL); } } else { @@ -394,7 +380,6 @@ printf_doformat(char *fmt, int *rval) start[strlen(start) - 1] = 's'; if ((p = strdup(getstr())) == NULL) { warnx("%s", strerror(ENOMEM)); - free(start); return (NULL); } getout = escape(p, 0, &len); @@ -402,10 +387,8 @@ printf_doformat(char *fmt, int *rval) /* Restore format for next loop. */ free(p); - if (getout) { - free(start); + if (getout) return (end_fmt); - } break; } case 'c': { @@ -430,10 +413,8 @@ printf_doformat(char *fmt, int *rval) int signedconv; signedconv = (convch == 'd' || convch == 'i'); - if ((f = mknum(start, convch)) == NULL) { - free(start); + if ((f = mknum(start, convch)) == NULL) return (NULL); - } if (getnum(&val, &uval, signedconv)) *rval = 1; if (signedconv) @@ -458,11 +439,9 @@ printf_doformat(char *fmt, int *rval) } default: warnx("illegal format character %c", convch); - free(start); return (NULL); } *fmt = nextch; - free(start); /* return the gargv to the next element */ return (fmt); } -- 2.32.0 From 838c1faef61b2af96c5d35cdb74661eac01e880c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 5 Nov 2020 17:49:22 +0100 Subject: [PATCH 238/325] strip-freebsd.sh: Ditch qat firmware files if available Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index a420f0103f25..0d0b5c11de6d 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -37,6 +37,14 @@ get_dev_ice_files() { fi } +# qat firmware modules were added in r367387 and are not (yet) +# available in stable/11. +get_dev_qatfw_files() { + if [ -d sys/contrib/dev/qat ]; then + find sys/contrib/dev/qat + fi +} + get_snd_csa_files() { find sys/dev/sound/pci -name "csa*" } @@ -69,6 +77,7 @@ get_usb_firmware_files() { get_unfree_files() { find sys/ -name "*.uu" get_dev_ice_files + get_dev_qatfw_files get_snd_csa_files get_snd_ds1_files get_snd_maestro3_files -- 2.32.0 From 77319aac290bd3c6503af711784c859ae68296f3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 26 Nov 2020 11:12:11 +0100 Subject: [PATCH 239/325] strip-freebsd.sh: Ditch imx firmware if available Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 0d0b5c11de6d..b64646b593f3 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -37,6 +37,14 @@ get_dev_ice_files() { fi } +# imx firmware was added in r368042 and is not (yet) +# available in stable/11. +get_dev_imx_files() { + if [ -d sys/contrib/dev/imx ]; then + find sys/contrib/dev/imx + fi +} + # qat firmware modules were added in r367387 and are not (yet) # available in stable/11. get_dev_qatfw_files() { @@ -77,6 +85,7 @@ get_usb_firmware_files() { get_unfree_files() { find sys/ -name "*.uu" get_dev_ice_files + get_dev_imx_files get_dev_qatfw_files get_snd_csa_files get_snd_ds1_files -- 2.32.0 From b5571ec5d16287aaf1e8fac992799ea1692ec418 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 17 Dec 2020 18:45:26 +0100 Subject: [PATCH 240/325] share/mk/src.opts.mk: Attach LIB32 to the build again ... for testing purposes as it is required by virtualbox. This reverts "share/mk/src.opts.mk: Detach LIB32 from the build" Obtained from: ElectroBSD --- share/mk/src.opts.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index b1732d245c1a..0015edb392a8 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -114,6 +114,7 @@ __DEFAULT_YES_OPTIONS = \ LDNS \ LDNS_UTILS \ LEGACY_CONSOLE \ + LIB32 \ LIBPTHREAD \ LIBTHR \ LLVM_COV \ -- 2.32.0 From 46fb3a567a34c5cc9d4af4469b53752d34a25c54 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Feb 2021 13:14:23 +0100 Subject: [PATCH 241/325] tools/sched/schedgraph.py: start conversion to python 3 Obtained from: ElectroBSD --- tools/sched/schedgraph.py | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) mode change 100644 => 100755 tools/sched/schedgraph.py diff --git a/tools/sched/schedgraph.py b/tools/sched/schedgraph.py old mode 100644 new mode 100755 index 6d8a4b9ada14..298f3a0774f9 --- a/tools/sched/schedgraph.py +++ b/tools/sched/schedgraph.py @@ -26,11 +26,10 @@ # # $FreeBSD$ -from __future__ import print_function import sys import re import random -from Tkinter import * +from tkinter import * # To use: # - Install the ports/x11-toolkits/py-tkinter package; e.g. @@ -470,7 +469,7 @@ def __init__(self, source): else: eventtypes[event.name] = (1, duration) events = [] - for k, v in eventtypes.iteritems(): + for k, v in eventtypes.items(): (c, d) = v events.append((k, c, d)) events.sort(cmp=cmp_counts) @@ -609,11 +608,11 @@ def bpress(self): self.newevent(prev) def fpress(self): - next = self.event.next() + next = next(self.event) if (next == None): return while (next.type == "pad"): - next = next.next() + next = next(next) if (next == None): return self.newevent(next) -- 2.32.0 From c1d0fff78c1489f70c6b696a0ced351b1d1a8008 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Feb 2021 13:20:17 +0100 Subject: [PATCH 242/325] tools/sched/schedgraph.py: implement __lt__ for EventSource so sorting works. XXX: is coparing y correct? Obtained from: ElectroBSD --- tools/sched/schedgraph.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/sched/schedgraph.py b/tools/sched/schedgraph.py index 298f3a0774f9..233492dea422 100755 --- a/tools/sched/schedgraph.py +++ b/tools/sched/schedgraph.py @@ -814,6 +814,9 @@ def __cmp__(self, other): return cmp(self.name, other.name) return cmp(self.group, other.group) + def __lt__(self, other): + return self.y < other.y + # It is much faster to append items to a list then to insert them # at the beginning. As a result, we add events in reverse order # and then swap the list during fixup. -- 2.32.0 From 7c11235e7ec6e55479babe05466d006d47e753ea Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 Oct 2021 14:41:05 +0200 Subject: [PATCH 243/325] sys/amd64/conf/ELECTRO_BLOAT: Sync with stable/12 GENERIC Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 54 +++++++++++++++++++++++++----------- 1 file changed, 38 insertions(+), 16 deletions(-) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index 8016f8898070..680d287e26ad 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -28,10 +28,15 @@ makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support options SCHED_ULE # ULE scheduler +options NUMA # Non-Uniform Memory Architecture support options PREEMPTION # Enable kernel thread preemption +options VIMAGE # Subsystem virtualization, e.g. VNET options INET # InterNETworking options INET6 # IPv6 communications protocols options TCP_OFFLOAD # TCP offload +options TCP_BLACKBOX # Enhanced TCP event logging +options TCP_HHOOK # hhook(9) framework for TCP +options TCP_RFC7413 # TCP Fast Open options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists @@ -54,8 +59,10 @@ options GEOM_PART_GPT # GUID Partition Tables. # affects other geom classes as well. nooptions GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization +options EFIRT # EFI Runtime Services support options COMPAT_FREEBSD32 # Compatible with i386 binaries options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options COMPAT_FREEBSD11 # Compatible with FreeBSD11 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options STACK # stack(9) support @@ -81,6 +88,13 @@ options RCTL # Resource limits # Debugging support. Always need this: options KDB # Enable kernel debugger support. options KDB_TRACE # Print a stack trace for a panic. + +# Kernel dump features. +options EKCD # Support for encrypted kernel dumps +options GZIO # gzip-compressed kernel and user dumps +options ZSTDIO # zstd-compressed kernel and user dumps +options NETDUMP # netdump(4) client support + # For full debugger support use (turn off in stable branch): options DDB # Support DDB. options GDB # Support remote GDB. @@ -89,6 +103,7 @@ options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones # Make an SMP-capable kernel by default options SMP # Symmetric MultiProcessor Kernel +options EARLY_AP_STARTUP # CPU frequency control device cpufreq @@ -97,22 +112,18 @@ device cpufreq device acpi options ACPI_DMAR device pci +options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers -options ATA_STATIC_ID # Static device numbering device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA # SCSI Controllers device ahc # AHA2940 and onboard AIC7xxx devices -options AHC_REG_PRETTY_PRINT # Print register bitfields in debug - # output. Adds ~128k to driver. device ahd # AHA39320/29320 and onboard AIC79xx devices -options AHD_REG_PRETTY_PRINT # Print register bitfields in debug - # output. Adds ~215k to driver. device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series device mpt # LSI-Logic MPT-Fusion @@ -201,14 +212,18 @@ device ppi # Parallel port interface device device puc # Multi I/O cards and multi-channel UARTs -# PCI Ethernet NICs. -device de # DEC/Intel DC21x4x (``Tulip'') +# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure +device iflib device em # Intel PRO/1000 Gigabit Ethernet Family -device igb # Intel PRO/1000 PCIE Server Gigabit Family +device igc # Intel I225 2.5G Ethernet device ix # Intel PRO/10GbE PCIE PF Ethernet device ixv # Intel PRO/10GbE PCIE VF Ethernet -device ixl # Intel XL710 40Gbe PCIE Ethernet -device ixlv # Intel XL710 40Gbe VF PCIE Ethernet +device ixl # Intel 700 Series Physical Function +device iavf # Intel Adaptive Virtual Function +device vmx # VMware VMXNET3 Ethernet + +# PCI Ethernet NICs. +device de # DEC/Intel DC21x4x (``Tulip'') device le # AMD Am7900 LANCE and Am79C9xx PCnet device ti # Alteon Networks Tigon I/II gigabit Ethernet device txp # 3Com 3cR990 (``Typhoon'') @@ -272,16 +287,18 @@ device ath_rate_sample # SampleRate tx rate control for ath device malo # Marvell Libertas wireless NICs. # Pseudo devices. +device crypto # core crypto support device loop # Network loopback device random # Entropy device device padlock_rng # VIA Padlock RNG device rdrand_rng # Intel Bull Mountain RNG device ether # Ethernet support device vlan # 802.1Q VLAN support -device tun # Packet tunnel. +device tuntap # Packet tunnel. device md # Memory "disks" device gif # IPv6 and IPv4 tunneling device firmware # firmware assist module +device xz # lzma decompression # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! @@ -320,7 +337,10 @@ device virtio_blk # VirtIO Block device device virtio_scsi # VirtIO SCSI device device virtio_balloon # VirtIO Memory Balloon device -# HyperV drivers and enchancement support +# Linux KVM paravirtualization support +device kvm_clock # KVM paravirtual clock driver + +# HyperV drivers and enhancement support device hyperv # HyperV drivers # Xen HVM Guest Optimizations @@ -328,8 +348,10 @@ device hyperv # HyperV drivers options XENHVM # Xen HVM kernel infrastructure device xenpci # Xen HVM Hypervisor services driver -# VMware support -device vmx # VMware VMXNET3 Ethernet +# Netmap provides direct access to TX/RX rings on supported NICs +device netmap # netmap(4) support -# The crypto framework is required by IPSEC -device crypto # Required by IPSEC +# evdev interface +options EVDEV_SUPPORT # evdev support in legacy drivers +device evdev # input event device support +device uinput # install /dev/uinput cdev -- 2.32.0 From fa24fd9af20c69fac715d72df279a4dacf7c6b38 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 26 Oct 2021 15:46:21 +0200 Subject: [PATCH 244/325] share/mk/src.opts.mk: Fix build. Squash Obtained from: ElectroBSD --- share/mk/src.opts.mk | 3 ++- sys/conf/kern.opts.mk | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 0015edb392a8..00d6c9359b2c 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -137,7 +137,6 @@ __DEFAULT_YES_OPTIONS = \ NLS_CATALOGS \ NS_CACHING \ NTP \ - OFED \ OPENSSL \ PAM \ PF \ @@ -148,6 +147,7 @@ __DEFAULT_YES_OPTIONS = \ RESCUE \ RCS \ RESCUE \ + SERVICESDB \ SETUID_LOGIN \ SYSCONS \ SYSTEM_COMPILER \ @@ -245,6 +245,7 @@ __DEFAULT_NO_OPTIONS += \ PORTSNAP \ PPP \ RCMDS \ + ROUTED \ SENDMAIL \ SVNLITE \ SHAREDOCS \ diff --git a/sys/conf/kern.opts.mk b/sys/conf/kern.opts.mk index 8abf76de7068..c60311a8486b 100644 --- a/sys/conf/kern.opts.mk +++ b/sys/conf/kern.opts.mk @@ -53,7 +53,7 @@ __DEFAULT_NO_OPTIONS = \ NAND \ OFED \ RATELIMIT \ - REPRODUCIBLE_BUILD \ + SCTP_SUPPORT \ SOURCELESS_HOST \ SOURCELESS_UCODE -- 2.32.0 From 468530aa9230681ba01e0bb54e1d3e08d3c169f0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 27 Oct 2021 17:15:22 +0200 Subject: [PATCH 245/325] sys/modules/Makefile: Fix build. Borked in upstream commit e0e2f49ed3? Obtained from: ElectroBSD --- sys/modules/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index 23b7ea0ef555..1653cbd17437 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -472,7 +472,7 @@ _ipfw_pmod= ipfw_pmod .if ${KERN_OPTS:MIPSEC_SUPPORT} && !${KERN_OPTS:MIPSEC} _ipsec= ipsec .endif -.if ${MK_SCTP_SUPPORT} != "no" || ${MK_SCTP} != "no" +.if ${MK_SCTP_SUPPORT} != "no" _sctp= sctp .endif .endif -- 2.32.0 From 5732597d931188135420b5ef0fa52bd97545cee1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 27 Oct 2021 17:47:55 +0200 Subject: [PATCH 246/325] release/scripts/strip-freebsd.sh: Update comment about ice. Squash Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index b64646b593f3..556b2ba4e365 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -25,9 +25,8 @@ # likely to still contain lots of non-free parts that haven't # been discovered yet. -# ice was added in r361541/ba15208e9900 and is not (yet) -# available in stable/11. It relies on a proprietary -# "DDP package file" (sys/contrib/dev/ice/ice-1.3.9.0.pkg) +# ice relies on a proprietary "DDP package file" +# (sys/contrib/dev/ice/ice-1.3.9.0.pkg) get_dev_ice_files() { if [ -d sys/contrib/dev/ice ]; then find sys/contrib/dev/ice -- 2.32.0 From 00230ed59aec32fef41bc4c58b0e9f339fd112f1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 27 Oct 2021 19:06:29 +0200 Subject: [PATCH 247/325] sys/modules/Makefile: Don't try to build ice which relies on proprietary firmware Obtained from: ElectroBSD --- sys/modules/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index 1653cbd17437..67acc3369a1e 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -576,8 +576,8 @@ _cxgbe= cxgbe .endif .if ${MACHINE_ARCH} == "amd64" || ${MACHINE_ARCH} == "arm64" -_ice= ice .if ${MK_SOURCELESS_UCODE} != "no" +_ice= ice _ice_ddp= ice_ddp .endif .endif -- 2.32.0 From bc87c6e2da6cef96793e62e78875e3e3781ed1b1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 27 Oct 2021 19:28:57 +0200 Subject: [PATCH 248/325] sys/modules/Makefile: Only build qatfw if MK_SOURCELESS_UCODE is enabled Obtained from: ElectroBSD --- sys/modules/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/modules/Makefile b/sys/modules/Makefile index 67acc3369a1e..9aa16f2bd725 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -667,7 +667,9 @@ _ndis= ndis _ocs_fc= ocs_fc _pccard= pccard _qat= qat +.if ${MK_SOURCELESS_UCODE} != "no" _qatfw= qatfw +.endif .if ${MK_OFED} != "no" || defined(ALL_MODULES) _rdma= rdma .endif -- 2.32.0 From 6aeb0ec320b28d4d00917fa158182690c521b908 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 28 Oct 2021 15:29:16 +0200 Subject: [PATCH 249/325] sys/amd64/conf/ELECTRO_BLOAT: Bring back COMPAT_LINUXKPI based on the assumption that it's needed for drm-kmod Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/amd64/conf/ELECTRO_BLOAT b/sys/amd64/conf/ELECTRO_BLOAT index 680d287e26ad..eb0fd9cf0491 100644 --- a/sys/amd64/conf/ELECTRO_BLOAT +++ b/sys/amd64/conf/ELECTRO_BLOAT @@ -115,6 +115,8 @@ device pci options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support +options COMPAT_LINUXKPI + # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers -- 2.32.0 From 9d285aa24a01a129604006ea962e42a0575b5bae Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 29 Oct 2021 16:14:26 +0200 Subject: [PATCH 250/325] Relocate soft-protection rc script. Squash Obtained from: ElectroBSD --- libexec/rc/rc.d/Makefile | 1 + {etc => libexec/rc}/rc.d/soft-protection | 0 2 files changed, 1 insertion(+) rename {etc => libexec/rc}/rc.d/soft-protection (100%) diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index 1f0ffed41e14..24ff069b41df 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -99,6 +99,7 @@ CONFS= DAEMON \ savecore \ securelevel \ serial \ + soft-protection \ statd \ static_arp \ static_ndp \ diff --git a/etc/rc.d/soft-protection b/libexec/rc/rc.d/soft-protection similarity index 100% rename from etc/rc.d/soft-protection rename to libexec/rc/rc.d/soft-protection -- 2.32.0 From b9f4032527a63ba813b2e0dd45409e3d1c8f4cc5 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Nov 2021 17:20:36 +0100 Subject: [PATCH 251/325] libexec/rc/rc.d/Makefile: Detach os-release from the build for now (depends on freebsd-version) Obtained from: ElectroBSD --- libexec/rc/rc.d/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index 24ff069b41df..40bd186095a7 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -77,7 +77,6 @@ CONFS= DAEMON \ nsswitch \ ntpdate \ ${_opensm} \ - os-release \ pf \ pflog \ pfsync \ -- 2.32.0 From ca4ea16723c53d3be5131fbdb498660b3ad97f60 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Nov 2021 19:14:40 +0100 Subject: [PATCH 252/325] sys/kern: Prevent inlining of chroot_refuse_vdir_fds() Obtained from: ElectroBSD --- sys/kern/kern_descrip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/kern_descrip.c b/sys/kern/kern_descrip.c index 5dc65d2d7723..5c4eb530bd50 100644 --- a/sys/kern/kern_descrip.c +++ b/sys/kern/kern_descrip.c @@ -3219,7 +3219,7 @@ SYSCTL_INT(_kern, OID_AUTO, chroot_allow_open_directories, CTLFLAG_RW, * Helper function for raised chroot(2) security function: Refuse if * any filedescriptors are open directories. */ -static int +static __noinline int chroot_refuse_vdir_fds(struct filedesc *fdp) { struct vnode *vp; -- 2.32.0 From caec967498b4e611f582ad50284ba822f1bbf44f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 3 Nov 2021 09:56:09 +0100 Subject: [PATCH 253/325] Bump kernel copyright Obtained from: ElectroBSD --- sys/sys/copyright.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/sys/copyright.h b/sys/sys/copyright.h index ef90ba2d44f7..48de0f68da9b 100644 --- a/sys/sys/copyright.h +++ b/sys/sys/copyright.h @@ -32,7 +32,7 @@ /* Add a FreeBSD vendor copyright here */ #define COPYRIGHT_Vendor \ - "Copyright (c) 2010-2020 Fabian Keil - IT-Beratung und Polizei-Erziehung\n" + "Copyright (c) 2010-2021 Fabian Keil - IT-Beratung und Polizei-Erziehung\n" /* FreeBSD */ #define COPYRIGHT_FreeBSD \ -- 2.32.0 From afe0a98a71cb6538022987a7f0dadb59fab11910 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 3 Nov 2021 09:58:07 +0100 Subject: [PATCH 254/325] Disable REPRODUCIBLE_BUILD for the kernel Obtained from: ElectroBSD --- sys/conf/kern.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/conf/kern.opts.mk b/sys/conf/kern.opts.mk index c60311a8486b..ad7a57762dbc 100644 --- a/sys/conf/kern.opts.mk +++ b/sys/conf/kern.opts.mk @@ -40,7 +40,6 @@ __DEFAULT_YES_OPTIONS = \ MODULE_DRM2 \ NETGRAPH \ PF \ - REPRODUCIBLE_BUILD \ TESTS \ USB_GADGET_EXAMPLES \ ZFS @@ -53,6 +52,7 @@ __DEFAULT_NO_OPTIONS = \ NAND \ OFED \ RATELIMIT \ + REPRODUCIBLE_BUILD \ SCTP_SUPPORT \ SOURCELESS_HOST \ SOURCELESS_UCODE -- 2.32.0 From 605c5bd9186df50465e207b043ab639b984dca02 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 3 Nov 2021 13:57:40 +0100 Subject: [PATCH 255/325] libc: Add ctime_iso() which returns a date like 2021-11-03 13:36:15 (to be used by zpool) --- contrib/tzcode/stdtime/localtime.c | 11 +++++++++++ include/time.h | 1 + lib/libc/stdtime/Symbol.map | 4 ++++ 3 files changed, 16 insertions(+) diff --git a/contrib/tzcode/stdtime/localtime.c b/contrib/tzcode/stdtime/localtime.c index 632a8d4a89ba..752194ccd7ae 100644 --- a/contrib/tzcode/stdtime/localtime.c +++ b/contrib/tzcode/stdtime/localtime.c @@ -1740,6 +1740,17 @@ ctime(const time_t *const timep) return asctime(localtime(timep)); } +static char buf_ctime_iso[21]; + +/* Print a date like 2021-11-03 13:36:15\n */ +char * +ctime_iso(const time_t *const timep) +{ + (void) strftime(buf_ctime_iso, sizeof(buf_ctime_iso), "%Y-%m-%d %H:%M:%S\n", localtime(timep)); + return buf_ctime_iso; +} + + char * ctime_r(const time_t *const timep, char *buf) { diff --git a/include/time.h b/include/time.h index 2a9bcb6a1107..a5d8e57bc711 100644 --- a/include/time.h +++ b/include/time.h @@ -148,6 +148,7 @@ __BEGIN_DECLS char *asctime(const struct tm *); clock_t clock(void); char *ctime(const time_t *); +char *ctime_iso(const time_t *); #ifndef _STANDALONE double difftime(time_t, time_t); #endif diff --git a/lib/libc/stdtime/Symbol.map b/lib/libc/stdtime/Symbol.map index 3bee3735f7e6..831f87411fc2 100644 --- a/lib/libc/stdtime/Symbol.map +++ b/lib/libc/stdtime/Symbol.map @@ -33,3 +33,7 @@ FBSD_1.0 { asctime_r; asctime; }; + +FBSD_1.1 { + ctime_iso; +}; -- 2.32.0 From 48d071d1f576bdb3f91361a61bd2a3e1dfdc241f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 3 Nov 2021 14:10:31 +0100 Subject: [PATCH 256/325] cddl/contrib/opensolaris/cmd/zpool/zpool_main.c: Use ctime_iso() instead of ctime() Obtained from: ElectroBSD --- .../opensolaris/cmd/zpool/zpool_main.c | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c b/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c index 35c3db7893df..e0c2ce3837a6 100644 --- a/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c +++ b/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c @@ -2320,7 +2320,7 @@ do_import(nvlist_t *config, const char *newname, const char *mntopts, "Last accessed by %s (hostid=%lx) at %s" "The pool can be imported, use 'zpool import -f' " "to import the pool.\n"), name, hostname, - (unsigned long)hostid, ctime((time_t *)×tamp)); + (unsigned long)hostid, ctime_iso((time_t *)×tamp)); } @@ -4781,14 +4781,14 @@ print_scan_status(pool_scan_stat_t *ps) "with %llu errors on %s"), processed_buf, (u_longlong_t)days_left, (u_longlong_t)hours_left, (u_longlong_t)mins_left, (u_longlong_t)secs_left, - (u_longlong_t)ps->pss_errors, ctime(&end)); + (u_longlong_t)ps->pss_errors, ctime_iso(&end)); } else if (ps->pss_func == POOL_SCAN_RESILVER) { (void) printf(gettext("resilvered %s " "in %llu days %02llu:%02llu:%02llu " "with %llu errors on %s"), processed_buf, (u_longlong_t)days_left, (u_longlong_t)hours_left, (u_longlong_t)mins_left, (u_longlong_t)secs_left, - (u_longlong_t)ps->pss_errors, ctime(&end)); + (u_longlong_t)ps->pss_errors, ctime_iso(&end)); } @@ -4796,10 +4796,10 @@ print_scan_status(pool_scan_stat_t *ps) } else if (ps->pss_state == DSS_CANCELED) { if (ps->pss_func == POOL_SCAN_SCRUB) { (void) printf(gettext("scrub canceled on %s"), - ctime(&end)); + ctime_iso(&end)); } else if (ps->pss_func == POOL_SCAN_RESILVER) { (void) printf(gettext("resilver canceled on %s"), - ctime(&end)); + ctime_iso(&end)); } return; } @@ -4810,16 +4810,16 @@ print_scan_status(pool_scan_stat_t *ps) if (ps->pss_func == POOL_SCAN_SCRUB) { if (pause == 0) { (void) printf(gettext("scrub in progress since %s"), - ctime(&start)); + ctime_iso(&start)); } else { (void) printf(gettext("scrub paused since %s"), - ctime(&pause)); + ctime_iso(&pause)); (void) printf(gettext("\tscrub started on %s"), - ctime(&start)); + ctime_iso(&start)); } } else if (ps->pss_func == POOL_SCAN_RESILVER) { (void) printf(gettext("resilver in progress since %s"), - ctime(&start)); + ctime_iso(&start)); } scanned = ps->pss_examined; @@ -4966,10 +4966,10 @@ print_removal_status(zpool_handle_t *zhp, pool_removal_stat_t *prs) copied_buf, (u_longlong_t)(minutes_taken / 60), (uint_t)(minutes_taken % 60), - ctime((time_t *)&end)); + ctime_iso((time_t *)&end)); } else if (prs->prs_state == DSS_CANCELED) { (void) printf(gettext("Removal of %s canceled on %s"), - vdev_name, ctime(&end)); + vdev_name, ctime_iso(&end)); } else { uint64_t copied, total, elapsed, mins_left, hours_left; double fraction_done; @@ -4982,7 +4982,7 @@ print_removal_status(zpool_handle_t *zhp, pool_removal_stat_t *prs) */ (void) printf(gettext( "Evacuation of %s in progress since %s"), - vdev_name, ctime(&start)); + vdev_name, ctime_iso(&start)); copied = prs->prs_copied > 0 ? prs->prs_copied : 1; total = prs->prs_to_copy; @@ -5040,10 +5040,10 @@ print_checkpoint_status(pool_checkpoint_stat_t *pcs) zfs_nicenum(pcs->pcs_space, space_buf, sizeof (space_buf)); if (pcs->pcs_state == CS_CHECKPOINT_EXISTS) { - char *date = ctime(&start); + char *date = ctime_iso(&start); /* - * ctime() adds a newline at the end of the generated + * ctime_iso() adds a newline at the end of the generated * string, thus the weird format specifier and the * strlen() call used to chop it off from the output. */ -- 2.32.0 From d0c5ee5859a941a9540f225b2197b33aeb14bd5c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 3 Nov 2021 17:40:05 +0100 Subject: [PATCH 257/325] ZFS: Use https for the error message URLs Obtained from: ElectroBSD --- cddl/contrib/opensolaris/cmd/zpool/zpool_main.c | 4 ++-- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c b/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c index e0c2ce3837a6..051ac7b6746b 100644 --- a/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c +++ b/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c @@ -2200,7 +2200,7 @@ show_import(nvlist_t *config) } if (msgid != NULL) - (void) printf(gettext(" see: http://illumos.org/msg/%s\n"), + (void) printf(gettext(" see: https://illumos.org/msg/%s\n"), msgid); (void) printf(gettext(" config:\n\n")); @@ -5440,7 +5440,7 @@ status_callback(zpool_handle_t *zhp, void *data) } if (msgid != NULL) - (void) printf(gettext(" see: http://illumos.org/msg/%s\n"), + (void) printf(gettext(" see: https://illumos.org/msg/%s\n"), msgid); if (config != NULL) { diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c index 1b452501819e..767561b3097d 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c @@ -2774,7 +2774,7 @@ spa_verify_host(spa_t *spa, nvlist_t *mos_config) cmn_err(CE_WARN, "pool '%s' could not be " "loaded as it was last accessed by " "another system (host: %s hostid: 0x%llx). " - "See: http://illumos.org/msg/ZFS-8000-EY", + "See: https://illumos.org/msg/ZFS-8000-EY", spa_name(spa), hostname, (u_longlong_t)hostid); spa_load_failed(spa, "hostid verification failed: pool " "last accessed by host: %s (hostid: 0x%llx)", -- 2.32.0 From d43f10585a106b026849c8e6e793d8b57db5b29f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 14:25:30 +0100 Subject: [PATCH 258/325] reproduce.sh: Attempt to unbreak the renaming of the release file directory Obtained from: ElectroBSD --- reproduce.sh | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/reproduce.sh b/reproduce.sh index a0cc4bf7c0c7..5c22de8493ff 100755 --- a/reproduce.sh +++ b/reproduce.sh @@ -73,13 +73,20 @@ announce_status() { reproduce_all_the_things() { local \ - f release_dir + f release_dir objdir # In a perfect world the umask shouldn't matter here but # in the real world it does for the single file /etc/rc.local # on the memstick. umask 022 - release_dir="${MAKEOBJDIRPREFIX}${SRC_DIR}/release" + + objdir="$(make -V .OBJDIR)" + if [ -z "${objdir}" ]; then + announce_status "Failed to get .OBJDIR ..." + fi + release_dir="${objdir}/release" + announce_status "Using release dir ${release_dir} ..." + if "${RESUME_BUILD}"; then announce_status "Resuming ..." export KERNFAST=1 -- 2.32.0 From c73c78cff007e870da3c0dcd8da3d45f9d74551c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 14:36:15 +0100 Subject: [PATCH 259/325] reproduce.sh: Bump copyright Obtained from: ElectroBSD --- reproduce.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reproduce.sh b/reproduce.sh index 5c22de8493ff..9b7044b90b6f 100755 --- a/reproduce.sh +++ b/reproduce.sh @@ -1,7 +1,7 @@ #!/bin/sh ########################################################################## -# Copyright (c) 2015-2016 Fabian Keil +# Copyright (c) 2015-2021 Fabian Keil # # Permission to use, copy, modify, and distribute this software for any # purpose with or without fee is hereby granted, provided that the above -- 2.32.0 From 203ed2281537967a0edbba217298e94469fb8a85 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 17:24:55 +0100 Subject: [PATCH 260/325] newsyslog.conf: Do not give world read permissions and don't compress stuff Obtained from: ElectroBSD --- usr.sbin/newsyslog/newsyslog.conf | 36 +++++++++++++------------------ 1 file changed, 15 insertions(+), 21 deletions(-) diff --git a/usr.sbin/newsyslog/newsyslog.conf b/usr.sbin/newsyslog/newsyslog.conf index 51ac310ed443..318401454a70 100644 --- a/usr.sbin/newsyslog/newsyslog.conf +++ b/usr.sbin/newsyslog/newsyslog.conf @@ -8,28 +8,22 @@ # is no process which needs to be signalled when a given log file is # rotated, then the entry for that file should include the 'N' flag. # -# Note: some sites will want to select more restrictive protections than the -# defaults. In particular, it may be desirable to switch many of the 644 -# entries to 640 or 600. For example, some sites will consider the -# contents of maillog, messages, and lpd-errs to be confidential. In the -# future, these defaults may change to more conservative ones. -# # logfilename [owner:group] mode count size when flags [/pid_file] [sig_num] -/var/log/all.log 600 7 * @T00 J -/var/log/auth.log 600 7 1000 @0101T JC -/var/log/console.log 600 5 1000 * J -/var/log/cron 600 3 1000 * JC -/var/log/daily.log 640 7 * @T00 JN -/var/log/debug.log 600 7 1000 * JC -/var/log/init.log 644 3 1000 * J -/var/log/kerberos.log 600 7 1000 * J -/var/log/maillog 640 7 * @T00 JC -/var/log/messages 644 5 1000 @0101T JC -/var/log/monthly.log 640 12 * $M1D0 JN -/var/log/devd.log 644 3 1000 * JC -/var/log/security 600 10 1000 * JC -/var/log/utx.log 644 3 * @01T05 B -/var/log/weekly.log 640 5 * $W6D0 JN +/var/log/all.log 600 7 * @T00 +/var/log/auth.log 600 7 1000 @0101T C +/var/log/console.log 600 5 1000 * +/var/log/cron 600 3 1000 * C +/var/log/daily.log 640 7 * @T00 N +/var/log/debug.log 600 7 1000 * C +/var/log/init.log 640 3 1000 * +/var/log/kerberos.log 600 7 1000 * +/var/log/maillog 640 7 * @T00 C +/var/log/messages 640 5 1000 @0101T C +/var/log/monthly.log 640 12 * $M1D0 N +/var/log/devd.log 640 3 1000 * C +/var/log/security 600 10 1000 * C +/var/log/utx.log 640 3 * @01T05 B +/var/log/weekly.log 640 5 * $W6D0 N /etc/newsyslog.conf.d/* /usr/local/etc/newsyslog.conf.d/* -- 2.32.0 From 742c76b9f67aa71e5133ef78a10a2fe14e1591f1 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 16:27:31 +0100 Subject: [PATCH 261/325] sbin/geom/misc: Add g_metadata_search() Obtained from: ElectroBSD --- sbin/geom/misc/subr.c | 73 +++++++++++++++++++++++++++++++++++++++++++ sbin/geom/misc/subr.h | 3 ++ 2 files changed, 76 insertions(+) diff --git a/sbin/geom/misc/subr.c b/sbin/geom/misc/subr.c index 3985ae56edc6..eb25c484a94d 100644 --- a/sbin/geom/misc/subr.c +++ b/sbin/geom/misc/subr.c @@ -273,6 +273,79 @@ g_metadata_read(const char *name, unsigned char *md, size_t size, return (error); } +int +g_metadata_search(const char *name, unsigned char *md, size_t size, + const char *magic) +{ + struct std_metadata stdmd; + unsigned char *sector; + ssize_t sectorsize; + off_t mediasize; + int error, fd; + off_t offset; + + sector = NULL; + error = 0; + + fd = g_open(name, 0); + if (fd == -1) + return (errno); + mediasize = g_mediasize(fd); + if (mediasize == -1) { + error = errno; + goto out; + } + sectorsize = g_sectorsize(fd); + if (sectorsize == -1) { + error = errno; + goto out; + } + assert(sectorsize >= (ssize_t)size); + sector = malloc(sectorsize); + if (sector == NULL) { + error = ENOMEM; + goto out; + } + + fprintf(stderr, "Searching for %s metadata on %s.\n", magic, name); + + offset = mediasize - sectorsize; + while (offset > 0) { + if (offset % 1073741824 == 0) { + /* Should be reworded */ + fprintf(stderr, + "Next search offset is %ju.\n", (uintmax_t)offset); + } + if (pread(fd, sector, sectorsize, offset) != + sectorsize) { + error = errno; + fprintf(stderr, "Failed to read at offset %ju.\n", + (uintmax_t)offset); + goto out; + } + if (magic != NULL) { + std_metadata_decode(sector, &stdmd); + if (strcmp(stdmd.md_magic, magic) != 0) { + offset -= sectorsize; + continue; + } else { + fprintf(stderr, + "Found %s meta data at offset %ju.\n", + magic, (uintmax_t)offset); + break; + } + } + offset -= sectorsize; + } + + bcopy(sector, md, size); +out: + if (sector != NULL) + free(sector); + g_close(fd); + return (error); +} + /* * Actually write the GEOM label to the provider * diff --git a/sbin/geom/misc/subr.h b/sbin/geom/misc/subr.h index 579c94c21491..80c27118db7b 100644 --- a/sbin/geom/misc/subr.h +++ b/sbin/geom/misc/subr.h @@ -41,6 +41,9 @@ unsigned int g_get_sectorsize(const char *name); int g_metadata_read(const char *name, unsigned char *md, size_t size, const char *magic); +int +g_metadata_search(const char *name, unsigned char *md, size_t size, + const char *magic); int g_metadata_store(const char *name, const unsigned char *md, size_t size); int g_metadata_clear(const char *name, const char *magic); -- 2.32.0 From 2c80b79f9856e08a62d7648f72bbb8eb847e7a09 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 4 Nov 2021 16:35:00 +0100 Subject: [PATCH 262/325] geli: Implement search command --- lib/geom/eli/geom_eli.c | 88 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/lib/geom/eli/geom_eli.c b/lib/geom/eli/geom_eli.c index bd1c60c40c96..a966ce6c0d70 100644 --- a/lib/geom/eli/geom_eli.c +++ b/lib/geom/eli/geom_eli.c @@ -84,6 +84,7 @@ static void eli_resize(struct gctl_req *req); static void eli_version(struct gctl_req *req); static void eli_clear(struct gctl_req *req); static void eli_dump(struct gctl_req *req); +static void eli_search(struct gctl_req *req); static int eli_backup_create(struct gctl_req *req, const char *prov, const char *file); @@ -109,6 +110,7 @@ static int eli_backup_create(struct gctl_req *req, const char *prov, * version [prov ...] * clear [-v] prov ... * dump [-v] prov ... + * search [-v] prov ... */ struct g_command class_commands[] = { { "init", G_FLAG_VERBOSE, eli_main, @@ -277,6 +279,9 @@ struct g_command class_commands[] = { { "dump", G_FLAG_VERBOSE, eli_main, G_NULL_OPTS, "[-v] prov ..." }, + { "search", G_FLAG_VERBOSE, eli_main, G_NULL_OPTS, + "[-v] prov ..." + }, G_CMD_SENTINEL }; @@ -339,6 +344,8 @@ eli_main(struct gctl_req *req, unsigned int flags) eli_restore(req); else if (strcmp(name, "resize") == 0) eli_resize(req); + else if (strcmp(name, "search") == 0) + eli_search(req); else if (strcmp(name, "version") == 0) eli_version(req); else if (strcmp(name, "dump") == 0) @@ -650,6 +657,62 @@ eli_metadata_read(struct gctl_req *req, const char *prov, return (0); } +static int +eli_metadata_search(struct gctl_req *req, const char *prov, + struct g_eli_metadata *md) +{ + unsigned char sector[sizeof(struct g_eli_metadata)]; + int error; + + if (g_get_sectorsize(prov) == 0) { + int fd; + + /* This is a file probably. */ + fd = open(prov, O_RDONLY); + if (fd == -1) { + gctl_error(req, "Cannot open %s: %s.", prov, + strerror(errno)); + return (-1); + } + if (read(fd, sector, sizeof(sector)) != sizeof(sector)) { + gctl_error(req, "Cannot read metadata from %s: %s.", + prov, strerror(errno)); + close(fd); + return (-1); + } + close(fd); + } else { + /* This is a GEOM provider. */ + error = g_metadata_search(prov, sector, sizeof(sector), + G_ELI_MAGIC); + if (error != 0) { + gctl_error(req, "Cannot read metadata from %s: %s.", + prov, strerror(error)); + return (-1); + } + } + error = eli_metadata_decode(sector, md); + switch (error) { + case 0: + break; + case EOPNOTSUPP: + gctl_error(req, + "Provider's %s metadata version %u is too new.\n" + "geli: The highest supported version is %u.", + prov, (unsigned int)md->md_version, G_ELI_VERSION); + return (-1); + case EINVAL: + gctl_error(req, "Inconsistent provider's %s metadata.", prov); + return (-1); + default: + gctl_error(req, + "Unexpected error while decoding provider's %s metadata: %s.", + prov, strerror(error)); + return (-1); + } + return (0); +} + static int eli_metadata_store(struct gctl_req *req, const char *prov, struct g_eli_metadata *md) @@ -1988,3 +2051,28 @@ eli_dump(struct gctl_req *req) printf("\n"); } } + +static void +eli_search(struct gctl_req *req) +{ + struct g_eli_metadata md; + const char *name; + int i, nargs; + + nargs = gctl_get_int(req, "nargs"); + if (nargs < 1) { + gctl_error(req, "Too few arguments."); + return; + } + + for (i = 0; i < nargs; i++) { + name = gctl_get_ascii(req, "arg%d", i); + if (eli_metadata_search(NULL, name, &md) == -1) { + gctl_error(req, "Not fully done."); + continue; + } + printf("Metadata found on %s:\n", name); + eli_metadata_dump(&md); + printf("\n"); + } +} -- 2.32.0 From 483ff307ac2a6783dead14946474b293bff02662 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 6 Nov 2021 09:24:35 +0100 Subject: [PATCH 263/325] share/mk/src.opts.mk: Disable LIB32 by default again (XXX: squash) Obtained from: ElectroBSD --- share/mk/src.opts.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index 00d6c9359b2c..736416c6362f 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -114,7 +114,6 @@ __DEFAULT_YES_OPTIONS = \ LDNS \ LDNS_UTILS \ LEGACY_CONSOLE \ - LIB32 \ LIBPTHREAD \ LIBTHR \ LLVM_COV \ @@ -181,6 +180,7 @@ __DEFAULT_NO_OPTIONS = \ LOADER_FORCE_LE \ LOADER_VERIEXEC_PASS_MANIFEST \ NAND \ + LIB32 \ OFED \ OFED_EXTRA \ OPENLDAP \ -- 2.32.0 From c98012dff972e52b8909c7db5a786cf51fd7f213 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 7 Nov 2021 11:45:01 +0100 Subject: [PATCH 264/325] g_eli: Don't include the thread number in the process name if there's only one thread Increases the chances that the provider isn't truncated as shown below (using ElectroBSD-20211105-d40b6832c298): last pid: 83498; load averages: 2.30, 1.84, 1.78 up 0+18:39:06 11:45:39 125 processes: 3 running, 121 sleeping, 1 waiting CPU: 42.2% user, 32.5% nice, 14.9% system, 7.6% interrupt, 2.7% idle Mem: 1125M Active, 406M Inact, 108M Laundry, 2092M Wired, 131M Free ARC: 1020M Total, 365M MFU, 376M MRU, 14M Anon, 17M Header, 248M Other, 1017M Target 468M Compressed, 658M Uncompressed, 1.41:1 Ratio Swap: 16G Total, 294M Used, 16G Free, 1% Inuse PID JID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 4126 4 _tor 4 93 0 1099M 690M CPU1 1 812:03 68.34% /usr/local/bin/tor -f /usr/local/etc/tor/torrc --PidFile /var/run/tor/tor.pid --RunAsDaemon 1 --DataDirectory /var/db/tor 33494 10 _tor 4 37 0 795M 392M kqread 0 296:39 24.45% /usr/local/bin/tor -f /usr/local/etc/tor/torrc --PidFile /var/run/tor/tor.pid --RunAsDaemon 1 --DataDirectory /var/db/tor 12 0 root 18 -52 - 0B 288K WAIT 1 166:22 14.37% [intr] 11 0 root 2 155 ki31 0B 32K RUN 0 549:39 5.24% [idle] 28 0 root 317 -16 - 0B 5088K spa->s 0 2:06 3.89% [zpool-rpool] 25 0 root 1 20 - 0B 16K geli:w 0 1:20 2.30% [g_eli[0] gpt/rpool-] 24 0 root 1 20 - 0B 16K geli:w 1 1:20 2.30% [g_eli[0] gpt/rpool-] 27 0 root 1 20 - 0B 16K geli:w 1 1:20 2.29% [g_eli[0] gpt/rpool-] 26 0 root 1 20 - 0B 16K geli:w 1 1:20 2.29% [g_eli[0] gpt/rpool-] 62912 0 root 1 20 - 0B 16K geli:w 0 18:15 0.92% [g_eli[0] gpt/dpool-] 26878 0 root 1 20 - 0B 16K geli:w 1 18:16 0.91% [g_eli[0] gpt/dpool-] 97173 0 root 1 20 - 0B 16K geli:w 1 18:16 0.90% [g_eli[0] gpt/dpool-] 17460 0 root 1 20 - 0B 16K geli:w 1 18:17 0.88% [g_eli[0] gpt/dpool-] Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 5aaaf44c92ad..1d4cc14c9f7d 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -888,9 +888,14 @@ g_eli_create(struct gctl_req *req, struct g_class *mp, struct g_provider *bpp, } goto failed; } - - error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, - "g_eli[%u] %s", i, bpp->name); + /* Only include the thread number if there's more than one */ + if (threads == 1) { + error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, + "g_eli %s", bpp->name); + } else { + error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, + "g_eli[%u] %s", i, bpp->name); + } if (error != 0) { g_eli_freesession(wr); free(wr, M_ELI); -- 2.32.0 From 193a74cfe1f965d0959ae976d153fead8c06b493 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 7 Nov 2021 19:54:30 +0100 Subject: [PATCH 265/325] sys/conf/newvers.sh: Ditch perforce support Obtained from: ElectroBSD --- sys/conf/newvers.sh | 29 ++--------------------------- 1 file changed, 2 insertions(+), 27 deletions(-) diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh index 5bf092a9a8d5..0bdc06f7f7c7 100755 --- a/sys/conf/newvers.sh +++ b/sys/conf/newvers.sh @@ -173,12 +173,6 @@ if [ -z "${svnversion}" ] && [ -x /usr/bin/svnliteversion ] ; then fi fi -for dir in /usr/bin /usr/local/bin; do - if [ -x "${dir}/p4" ] && [ -z ${p4_cmd} ] ; then - p4_cmd=${dir}/p4 - fi -done - if [ -z "${KERNEL_VERSION_NUMBER}" ] && findvcs .git; then for dir in /usr/bin /usr/local/bin; do if [ -x "${dir}/git" ] ; then @@ -231,25 +225,6 @@ if [ -n "$git_cmd" ] ; then git=" ${git}" fi -if [ -n "$p4_cmd" ] ; then - p4version=`cd ${SYSDIR} && $p4_cmd changes -m1 "./...#have" 2>&1 | \ - awk '{ print $2 }'` - case "$p4version" in - [0-9]*) - p4version=" ${p4version}" - p4opened=`cd ${SYSDIR} && $p4_cmd opened ./... 2>&1` - case "$p4opened" in - File*) ;; - //*) - p4version="${p4version}+edit" - modified=true - ;; - esac - ;; - *) unset p4version ;; - esac -fi - if [ -n "$hg_cmd" ] ; then hg=`$hg_cmd id 2>/dev/null` svn=`$hg_cmd svn info 2>/dev/null | \ @@ -277,10 +252,10 @@ done shift $((OPTIND - 1)) if [ -z "${include_metadata}" ]; then - VERINFO="${VERSION}${svn}${git}${hg}${p4version} ${i}" + VERINFO="${VERSION}${svn}${git}${hg} ${i}" VERSTR="${VERINFO}\\n" else - VERINFO="${VERSION} #${v}${svn}${git}${hg}${p4version}: ${t}" + VERINFO="${VERSION} #${v}${svn}${git}${hg}: ${t}" VERSTR="${VERINFO}\\n ${u}@${h}:${d}\\n" fi -- 2.32.0 From 61831f9d85d5a9c3e07f580c5096c27060a87969 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 28 Oct 2021 14:34:36 +0200 Subject: [PATCH 266/325] release/amd64/make-memstick.sh: Temporarily stop using the freshly build makefs May prevent: sh /usr/src/release/amd64/make-memstick.sh disc1 memstick.img /usr/obj/usr/src/amd64.amd64/release/dist/base/usr/sbin/makefs: Undefined symbol "stat@FBSD_1.5" *** Error code 1 Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index 7023f8b0a0fc..b713337fe579 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -50,7 +50,7 @@ MAKEFS_DEBUG_FLAGS=0x81400 # # Bind makefs to a single cpu as the output is core dependant # (XXX: doesn't seem to have the intended effect) -cpuset -l 0 dist/base/usr/sbin/makefs -T "${EPOCH_DATE-0}" \ +cpuset -l 0 makefs -T "${EPOCH_DATE-0}" \ -d "${MAKEFS_DEBUG_FLAGS}" \ -B little -o label="${VOLUME_LABEL}" -o version=2 -F "${2}.mtree" ${2}.part ${1} rm ${1}/etc/fstab -- 2.32.0 From 0469f5f63864788b528600f2388992557e346c35 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 28 Oct 2021 14:42:00 +0200 Subject: [PATCH 267/325] release/amd64/make-memstick.sh: Temporarily stop using the freshly build mkimg Obtained from: ElectroBSD --- release/amd64/make-memstick.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/amd64/make-memstick.sh b/release/amd64/make-memstick.sh index b713337fe579..f6ec9097ba08 100755 --- a/release/amd64/make-memstick.sh +++ b/release/amd64/make-memstick.sh @@ -57,7 +57,7 @@ rm ${1}/etc/fstab rm ${1}/etc/rc.conf.local rm "${2}.mtree" -dist/base/usr/bin/mkimg -s mbr \ +mkimg -s mbr \ -b ${1}/boot/mbr \ -p efi:=${1}/boot/boot1.efifat \ -p freebsd:-"mkimg -s bsd -b ${1}/boot/boot -p freebsd-ufs:=${2}.part" \ -- 2.32.0 From 64dfd7e6456ac023ef304695fba88a4ef408869d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 09:16:17 +0100 Subject: [PATCH 268/325] share/man/man5: Don't install os-release.5 which isn't currently available on ElectroBSD Obtained from: ElectroBSD --- share/man/man5/Makefile | 1 - 1 file changed, 1 deletion(-) diff --git a/share/man/man5/Makefile b/share/man/man5/Makefile index c353f10fed95..8e40279bec33 100644 --- a/share/man/man5/Makefile +++ b/share/man/man5/Makefile @@ -48,7 +48,6 @@ MAN= acct.5 \ nsmb.conf.5 \ nsswitch.conf.5 \ nullfs.5 \ - os-release.5 \ passwd.5 \ pbm.5 \ periodic.conf.5 \ -- 2.32.0 From 34ac6fae288b7745f73f978f6e0a62f25f3570e2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 09:16:58 +0100 Subject: [PATCH 269/325] etc: Don't create a /etc/os-release symlink as the os-release file generation currently doesn't work Obtained from: ElectroBSD --- etc/Makefile | 2 -- 1 file changed, 2 deletions(-) diff --git a/etc/Makefile b/etc/Makefile index f485728b7967..b9e9da60e4cc 100644 --- a/etc/Makefile +++ b/etc/Makefile @@ -82,8 +82,6 @@ distribution: ${_+_}cd ${.CURDIR}/mtree; ${MAKE} install ${_+_}cd ${SRCTOP}/share/termcap; ${MAKE} etc-termcap ${_+_}cd ${SRCTOP}/usr.sbin/rmt; ${MAKE} etc-rmt - ${INSTALL_SYMLINK} ../var/run/os-release \ - ${DESTDIR}/etc/os-release .if ${MK_UNBOUND} != "no" if [ ! -e ${DESTDIR}/etc/unbound ]; then \ ${INSTALL_SYMLINK} ../var/unbound ${DESTDIR}/etc/unbound; \ -- 2.32.0 From 1c707d59ee9f1458e4b9c1e4442fb0a81b1558b0 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 11:21:34 +0100 Subject: [PATCH 270/325] syslogd: Add ElectroBSD output format Obtained from: ElectroBSD --- usr.sbin/syslogd/syslogd.c | 68 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 66 insertions(+), 2 deletions(-) diff --git a/usr.sbin/syslogd/syslogd.c b/usr.sbin/syslogd/syslogd.c index 4b30c27e9e2d..fc6257a80edb 100644 --- a/usr.sbin/syslogd/syslogd.c +++ b/usr.sbin/syslogd/syslogd.c @@ -402,6 +402,7 @@ static int needdofsync = 0; /* Are any file(s) waiting to be fsynced? */ static struct pidfh *pfh; static int sigpipe[2]; /* Pipe to catch a signal during select(). */ static bool RFC3164OutputFormat = true; /* Use legacy format by default. */ +static bool ElectroBSDOutputFormat = false; static volatile sig_atomic_t MarkSet, WantDie, WantInitialize, WantReapchild; @@ -665,7 +666,10 @@ main(int argc, char *argv[]) resolve = 0; break; case 'O': - if (strcmp(optarg, "bsd") == 0 || + if (strcmp(optarg, "electrobsd") == 0) { + ElectroBSDOutputFormat = true; + RFC3164OutputFormat = false; + } else if (strcmp(optarg, "bsd") == 0 || strcmp(optarg, "rfc3164") == 0) RFC3164OutputFormat = true; else if (strcmp(optarg, "syslog") == 0 || @@ -1947,6 +1951,63 @@ fprintlog_rfc5424(struct filed *f, const char *hostname, const char *app_name, fprintlog_write(f, &il, flags); } + +/* + * Similiar to fprintlog_rfc5424() but the priority is moved + * after the app name so log messages always start with the + * timestamp and are more pleasant to read. + */ +static void +fprintlog_electrobsd(struct filed *f, const char *hostname, const char *app_name, + const char *procid, const char *msgid, const char *structured_data, + const char *msg, int flags) +{ + struct iovlist il; + suseconds_t usec; + int i; + char timebuf[33], priority_number[5]; + + iovlist_init(&il); + if (f->f_type == F_WALL) + iovlist_append(&il, "\r\n\aMessage from syslogd ...\r\n"); + if (strftime(timebuf, sizeof(timebuf), "%FT%T.______%z", + &f->f_lasttime.tm) == sizeof(timebuf) - 2) { + /* Add colon to the time zone offset, which %z doesn't do. */ + timebuf[32] = '\0'; + timebuf[31] = timebuf[30]; + timebuf[30] = timebuf[29]; + timebuf[29] = ':'; + + /* Overwrite space for microseconds with actual value. */ + usec = f->f_lasttime.usec; + for (i = 25; i >= 20; --i) { + timebuf[i] = usec % 10 + '0'; + usec /= 10; + } + iovlist_append(&il, timebuf); + } else + iovlist_append(&il, "-"); + iovlist_append(&il, " "); + iovlist_append(&il, hostname); + iovlist_append(&il, " "); + iovlist_append(&il, app_name == NULL ? "-" : app_name); + iovlist_append(&il, " "); + iovlist_append(&il, "<"); + snprintf(priority_number, sizeof(priority_number), "%d", f->f_prevpri); + iovlist_append(&il, priority_number); + iovlist_append(&il, ">1 "); + iovlist_append(&il, procid == NULL ? "-" : procid); + iovlist_append(&il, " "); + iovlist_append(&il, msgid == NULL ? "-" : msgid); + iovlist_append(&il, " "); + iovlist_append(&il, structured_data == NULL ? "-" : structured_data); + iovlist_append(&il, " "); + iovlist_append(&il, msg); + + fprintlog_write(f, &il, flags); +} + + static void fprintlog_rfc3164(struct filed *f, const char *hostname, const char *app_name, const char *procid, const char *msg, int flags) @@ -2071,7 +2132,10 @@ fprintlog_first(struct filed *f, const char *hostname, const char *app_name, return; } - if (RFC3164OutputFormat) + if (ElectroBSDOutputFormat) + fprintlog_electrobsd(f, hostname, app_name, procid, msgid, + structured_data, msg, flags); + else if (RFC3164OutputFormat) fprintlog_rfc3164(f, hostname, app_name, procid, msg, flags); else fprintlog_rfc5424(f, hostname, app_name, procid, msgid, -- 2.32.0 From 6def623e5dc903cb0cad93abb829e17c5929ec75 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 10:08:48 +0100 Subject: [PATCH 271/325] rc.conf: Let syslogd log in ElectroBSD format by default Obtained from: ElectroBSD --- libexec/rc/rc.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libexec/rc/rc.conf b/libexec/rc/rc.conf index e5a731d09e21..af0d93c4bb12 100755 --- a/libexec/rc/rc.conf +++ b/libexec/rc/rc.conf @@ -279,7 +279,7 @@ hostapd_program="/usr/sbin/hostapd" hostapd_enable="NO" # Run hostap daemon. syslogd_enable="YES" # Run syslog daemon (or NO). syslogd_program="/usr/sbin/syslogd" # path to syslogd, if you want a different one. -syslogd_flags="-s" # Flags to syslogd (if enabled). +syslogd_flags="-s -O electrobsd" # Flags to syslogd (if enabled). syslogd_oomprotect="YES" # Don't kill syslogd when swap space is exhausted. altlog_proglist="" # List of chrooted applicatioins in /var inetd_enable="NO" # Run the network daemon dispatcher (YES/NO). -- 2.32.0 From 4038a878f14fa6a5c32ac8072b5842b253e8e576 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 11:52:17 +0100 Subject: [PATCH 272/325] share/mk/bsd.own.mk: Use faster xz compression as we care more about time than space Obtained from: ElectroBSD --- share/mk/bsd.own.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/share/mk/bsd.own.mk b/share/mk/bsd.own.mk index 69db8c8eabca..d1e35424d4ac 100644 --- a/share/mk/bsd.own.mk +++ b/share/mk/bsd.own.mk @@ -259,9 +259,9 @@ COMPRESS_EXT?= .gz XZ_THREADS?= 0 .if !empty(XZ_THREADS) -XZ_CMD?= xz -T ${XZ_THREADS} +XZ_CMD?= xz -0 -T ${XZ_THREADS} .else -XZ_CMD?= xz +XZ_CMD?= xz -0 .endif PKG_CMD?= pkg -- 2.32.0 From 20b9aceb19a22f49c5f1ff4b4fbd7d36d7992b91 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 7 Nov 2021 15:08:32 +0100 Subject: [PATCH 273/325] sys/geom/eli: If the thread name will be truncated, use the last characters instead of the first ones Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index 1d4cc14c9f7d..e08812575693 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -786,7 +786,9 @@ g_eli_create(struct gctl_req *req, struct g_class *mp, struct g_provider *bpp, struct g_geom *gp; struct g_provider *pp; struct g_consumer *cp; - u_int i, threads; + struct thread dummy_thread; + u_int i, threads, name_offset; + size_t name_len, name_space_available; int dcw, error; G_ELI_DEBUG(1, "Creating device %s%s.", bpp->name, G_ELI_SUFFIX); @@ -888,13 +890,36 @@ g_eli_create(struct gctl_req *req, struct g_class *mp, struct g_provider *bpp, } goto failed; } - /* Only include the thread number if there's more than one */ + name_len = strlen(bpp->name); + name_space_available = sizeof(dummy_thread.td_name) - 1 - + strlen("g_eli"); + if (threads != 1) { + /* + * XXX: Assumes the number of threads + * fits a single digit + */ + name_space_available -= strlen("[0]"); + } + if (name_len < name_space_available) + name_offset = 0; + else + name_offset = name_len - name_space_available + 1; + + G_ELI_DEBUG(1, "geli threads for %s: %d, name_len: %d, " + "name_space_available: %d, name_offset: %d ", + bpp->name, threads, (int)name_len, + (int)name_space_available, name_offset); + + /* + * Only include the thread number if there's more + * than one thread. + */ if (threads == 1) { - error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, - "g_eli %s", bpp->name); + error = kproc_create(g_eli_worker, wr, &wr->w_proc, + 0, 0, "g_eli %s", &bpp->name[name_offset]); } else { - error = kproc_create(g_eli_worker, wr, &wr->w_proc, 0, 0, - "g_eli[%u] %s", i, bpp->name); + error = kproc_create(g_eli_worker, wr, &wr->w_proc, + 0, 0, "g_eli[%u] %s", i, &bpp->name[name_offset]); } if (error != 0) { g_eli_freesession(wr); -- 2.32.0 From 5804c9a1152cc0723e9a1d26bc42845f37b776d7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Nov 2021 18:02:40 +0100 Subject: [PATCH 274/325] release/Makefile: Let the memstick ignore geli providers with passphrase Obtained from: ElectroBSD --- release/Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/release/Makefile b/release/Makefile index 89dcccf86285..6ca2ea4ae19e 100644 --- a/release/Makefile +++ b/release/Makefile @@ -212,6 +212,9 @@ disc1: packagesystem echo hostname=\"${VOLUME_LABEL}\" >> ${.TARGET}/etc/rc.conf echo vfs.mountroot.timeout=\"10\" >> ${.TARGET}/boot/loader.conf echo kernels_autodetect=\"NO\" >> ${.TARGET}/boot/loader.conf + # Ignore provider with passphrase so booting doesn't require + # console interaction. + echo kern.geom.eli.tries=\"0\" >> ${.TARGET}/boot/loader.conf cp ${.CURDIR}/rc.local ${.TARGET}/etc umask ls -l ${.TARGET}/etc/rc.local -- 2.32.0 From 8c231e07404fafa48ab2ecd1c78d9bd8114a1f2e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Nov 2021 01:03:39 +0100 Subject: [PATCH 275/325] sbin/devd/zfs.conf: Include zpool name in vdev state change message Obtained from: ElectroBSD --- sbin/devd/zfs.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/devd/zfs.conf b/sbin/devd/zfs.conf index edabbbf3ed59..b447d97d70c1 100644 --- a/sbin/devd/zfs.conf +++ b/sbin/devd/zfs.conf @@ -72,6 +72,6 @@ notify 10 { notify 10 { match "system" "ZFS"; match "type" "resource.fs.zfs.statechange"; - action "logger -p local7.notice -t ZFS 'vdev state changed, pool_guid=$pool_guid vdev_guid=$vdev_guid'"; + action "logger -p local7.notice -t ZFS 'vdev state changed, zpool=$pool pool_guid=$pool_guid vdev_guid=$vdev_guid'"; }; -- 2.32.0 From f68cda40d6d266fc3da77cf25fd2182aef0db830 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Nov 2021 18:58:31 +0100 Subject: [PATCH 276/325] Revert "sbin/devd/zfs.conf: Include zpool name in vdev state change message" ... as it doesn't work yet. This reverts commit 90275b951521461db8f3a6b955e01bf1da62a7f1. --- sbin/devd/zfs.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/devd/zfs.conf b/sbin/devd/zfs.conf index b447d97d70c1..edabbbf3ed59 100644 --- a/sbin/devd/zfs.conf +++ b/sbin/devd/zfs.conf @@ -72,6 +72,6 @@ notify 10 { notify 10 { match "system" "ZFS"; match "type" "resource.fs.zfs.statechange"; - action "logger -p local7.notice -t ZFS 'vdev state changed, zpool=$pool pool_guid=$pool_guid vdev_guid=$vdev_guid'"; + action "logger -p local7.notice -t ZFS 'vdev state changed, pool_guid=$pool_guid vdev_guid=$vdev_guid'"; }; -- 2.32.0 From 3f6d4a462639a67e950cf9b3cb3e597ac79b6104 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Nov 2021 08:28:41 +0100 Subject: [PATCH 277/325] sys/geom/eli: For the last passphrase entering attempt, turn on visibility ... in case the keyboard is prelling. Obtained from: ElectroBSD --- sys/geom/eli/g_eli.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/sys/geom/eli/g_eli.c b/sys/geom/eli/g_eli.c index e08812575693..21e9f0e93d24 100644 --- a/sys/geom/eli/g_eli.c +++ b/sys/geom/eli/g_eli.c @@ -1219,8 +1219,14 @@ g_eli_taste(struct g_class *mp, struct g_provider *pp, int flags __unused) memcpy(passphrase, cached_passphrase, sizeof(passphrase)); } else { + showpass = g_eli_visible_passphrase; + if (!showpass && i == tries) { + G_ELI_DEBUG(0, "Last try for %s. " + "Turning visibility on!!!", + pp->name); + showpass = 1; + } printf("Enter passphrase for %s: ", pp->name); - showpass = g_eli_visible_passphrase; if ((md.md_flags & G_ELI_FLAG_GELIDISPLAYPASS) != 0) showpass = GETS_ECHOPASS; cngets(passphrase, sizeof(passphrase), -- 2.32.0 From 5c537a2cd604876004d18c62d1e28da156bf5999 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Nov 2021 16:05:14 +0100 Subject: [PATCH 278/325] Add a ELECTRO_BLOAT_BOOT_PROFILING kernel config Obtained from: ElectroBSD --- sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING diff --git a/sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING b/sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING new file mode 100644 index 000000000000..9447ffa7971d --- /dev/null +++ b/sys/amd64/conf/ELECTRO_BLOAT_BOOT_PROFILING @@ -0,0 +1,5 @@ +include ELECTRO_BLOAT + +ident ELECTRO_BLOAT_BOOT_PROFILING + +options TSLOG -- 2.32.0 From cbd14382efdd22da5c425700b5522ac3fff311e8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Nov 2021 18:29:08 +0100 Subject: [PATCH 279/325] reproduce.sh: Use time's -h flag to get more pleasant number. Squash. Obtained from: ElectroBSD --- reproduce.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/reproduce.sh b/reproduce.sh index 9b7044b90b6f..445566e43a38 100755 --- a/reproduce.sh +++ b/reproduce.sh @@ -97,10 +97,10 @@ reproduce_all_the_things() { # We build the world first, so the kernel is built # with a freshly built toolchain. announce_status "Starting to build the world" - time -l make -j${MAX_MAKE_JOBS} buildworld || return 1 + time -h -l make -j${MAX_MAKE_JOBS} buildworld || return 1 announce_status "Starting to build the kernel" - time -l make buildkernel || return 1 + time -h -l make buildkernel || return 1 # Make sure obj files aren't dumped in ${SRC_DIR} mkdir -p "${release_dir}" || return 1 @@ -110,7 +110,7 @@ reproduce_all_the_things() { make -C "${SRC_DIR}/release" clean fi announce_status "Starting to build the release" - time -l make -C "${SRC_DIR}/release" \ + time -h -l make -C "${SRC_DIR}/release" \ memstick XZ_THREADS=${XZ_THREADS} NO_FSCHG="yes" || return 1 announce_status "Done with release memstick for ${REPRO_SEED}" -- 2.32.0 From 543b064bf2ec5d959291c12fdc20249ee730beec Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 7 Nov 2021 17:35:51 +0100 Subject: [PATCH 280/325] Add sysctl to control whether or not pwd_chroot_chdir() calls chroot_refuse_vdir_fds() Obtained from: ElectroBSD --- sys/kern/kern_descrip.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/sys/kern/kern_descrip.c b/sys/kern/kern_descrip.c index 5c4eb530bd50..c2a64a45b61f 100644 --- a/sys/kern/kern_descrip.c +++ b/sys/kern/kern_descrip.c @@ -3290,6 +3290,12 @@ pwd_chdir(struct thread *td, struct vnode *vp) vrele(oldvp); } +static int pwd_chroot_chdir_check_open_directories = 1; + +SYSCTL_INT(_kern, OID_AUTO, pwd_chroot_chdir_check_open_directories, CTLFLAG_RW, + &pwd_chroot_chdir_check_open_directories, 0, + "Let pwd_chroot_chdir() check for open directories and fail if there are any"); + /* * jail_attach(2) changes both root and working directories. */ @@ -3302,10 +3308,12 @@ pwd_chroot_chdir(struct thread *td, struct vnode *vp) fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); - error = chroot_refuse_vdir_fds(fdp); - if (error != 0) { - FILEDESC_XUNLOCK(fdp); - return (error); + if (pwd_chroot_chdir_check_open_directories != 0) { + error = chroot_refuse_vdir_fds(fdp); + if (error != 0) { + FILEDESC_XUNLOCK(fdp); + return (error); + } } oldvrp = fdp->fd_rdir; vrefact(vp); -- 2.32.0 From 1f3085e65db880c6aa96268d62cc3f172d7ee53b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 27 Nov 2015 00:20:59 +0100 Subject: [PATCH 281/325] etc: Register uid and gid for Tor ... so the user doesn't have to do it manually on the host system after installing Tor in a jail. Obtained from: ElectroBSD --- etc/group | 1 + etc/master.passwd | 1 + 2 files changed, 2 insertions(+) diff --git a/etc/group b/etc/group index 9f24beda5aea..da928bf12c77 100644 --- a/etc/group +++ b/etc/group @@ -31,6 +31,7 @@ audit:*:77: www:*:80: ntpd:*:123: _ypldap:*:160: +_tor:*:256: hast:*:845: tests:*:977: nogroup:*:65533: diff --git a/etc/master.passwd b/etc/master.passwd index b61c13fdd9fe..597aebe125dc 100644 --- a/etc/master.passwd +++ b/etc/master.passwd @@ -24,6 +24,7 @@ auditdistd:*:78:77::0:0:Auditdistd unprivileged user:/var/empty:/usr/sbin/nologi www:*:80:80::0:0:World Wide Web Owner:/nonexistent:/usr/sbin/nologin ntpd:*:123:123::0:0:NTP Daemon:/var/db/ntp:/usr/sbin/nologin _ypldap:*:160:160::0:0:YP LDAP unprivileged user:/var/empty:/usr/sbin/nologin +_tor:*:256:256::0:0:Onion delivery agent:/nonexistent:/usr/sbin/nologin hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin tests:*:977:977::0:0:Unprivileged user for tests:/nonexistent:/usr/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin -- 2.32.0 From d4608f113f0ab4ca81225da0fd9882fc16b4ff7d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 11 Nov 2021 03:37:13 +0100 Subject: [PATCH 282/325] rc.conf: Default to syslogd not opening any network sockets Obtained from: ElectroBSD --- libexec/rc/rc.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libexec/rc/rc.conf b/libexec/rc/rc.conf index af0d93c4bb12..35369cac30ea 100755 --- a/libexec/rc/rc.conf +++ b/libexec/rc/rc.conf @@ -279,7 +279,7 @@ hostapd_program="/usr/sbin/hostapd" hostapd_enable="NO" # Run hostap daemon. syslogd_enable="YES" # Run syslog daemon (or NO). syslogd_program="/usr/sbin/syslogd" # path to syslogd, if you want a different one. -syslogd_flags="-s -O electrobsd" # Flags to syslogd (if enabled). +syslogd_flags="-ss -O electrobsd" # Flags to syslogd (if enabled). syslogd_oomprotect="YES" # Don't kill syslogd when swap space is exhausted. altlog_proglist="" # List of chrooted applicatioins in /var inetd_enable="NO" # Run the network daemon dispatcher (YES/NO). -- 2.32.0 From 13b8b0e5d3aca1e8d27416c8acf5d810d3258357 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 12 Nov 2021 03:56:15 +0100 Subject: [PATCH 283/325] geli(8): Document the search subcommand Obtained from: ElectroBSD --- lib/geom/eli/geli.8 | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index 23bad1999806..dace8bc68254 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd October 15, 2020 +.Dd November 12, 2021 .Dt GELI 8 .Os .Sh NAME @@ -91,6 +91,9 @@ utility: .Op Fl bBdDgGtT .Ar prov ... .Nm +.Cm search +.Ar prov +.Nm .Cm setkey .Op Fl pPv .Op Fl i Ar iterations @@ -547,6 +550,13 @@ subcommand. .It Fl T Disable TRIM/UNMAP passthru. .El +.It Cm search +Search for metadata on the provider, starting at the end and +going backwards until valid metadata is found or the beginning +of the provider is reached. +This subcommand may be useful if, for example, the GPT partition +data got corrupted or deleted while the data on the previously +accessible partitions is still expected to be valid. .It Cm setkey Install a copy of the Master Key into the selected slot, encrypted with a new User Key. -- 2.32.0 From 38696304d68b2525a27d6d2e9612bf598eb397ba Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 13 Sep 2017 04:31:10 +0200 Subject: [PATCH 284/325] ZFS: Add vfs.zfs.risk_data_corruption sysctl ... and don't split large blocks while sending unless the sysctl is set. Splitting large blocks can lead to silent data corruption when receiving incremental streams with a block size change on the receiving side. Example corruption: [fk@test-vm ~]$ hd /usr/test/src/sbin/camcontrol/camcontrol.c 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 0003ea10 [fk@test-vm ~]$ sudo zdb -ddddddd -bbbbbbb rpool/test/src $(stat -f '%i' /usr/test/src/sbin/camcontrol/camcontrol.c) Dataset rpool/test/src [ZPL], ID 550, cr_txg 67575, 704M, 78457 objects, rootbp DVA[0]=<0:254bfb000:1000> DVA[1]=<0:3803b7000:1000> [L0 DMU objset] sha256 uncompressed LE contiguous unique double size=800L/800P birth=67755L/67755P fill=78457 cksum=fb0fd08b71ffb3f8:e14c2e477498cdc4:4822440c7db2cccc:9fa4961bce0031b6 Object lvl iblk dblk dsize lsize %full type 100844 2 128K 251K 0 251K 0.00 ZFS plain file (K=inherit) (Z=inherit) 168 bonus System attributes dnode flags: USED_BYTES USERUSED_ACCOUNTED dnode maxblkid: 0 path /sbin/camcontrol/camcontrol.c uid 1001 gid 1001 atime Sat Jul 8 18:41:59 2017 mtime Sat Jul 8 18:41:59 2017 ctime Sat Jul 8 18:41:59 2017 crtime Sat Jul 8 18:41:59 2017 gen 609632 mode 100644 size 256529 parent 44643 links 1 pflags 40800000004 Indirect blocks: 0 L1 HOLE [L1 ZFS plain file] size=20000L birth=67755L segment [0000000000000000, 000000000003ec00) size 251K Note that the silent data corruption doesn't necessarily occur right away. For example splitting large blocks is fine when sending to a dataset with recordsize<=128k, but if the recordsize is later on increased to 1m the receiving of a incremental snapshot with large blocks (that haven't been split) can cause corruption on the receiving side as well. Never splitting large block at all avoids the (known and reproducible) problems. In practice it's a minor inconvenience and certainly less inconvenient than losing data. The zfs error message could be improved, though: # zfs send -i testpool/source@snap1 testpool/source@snap2 >/dev/null warning: cannot send 'testpool/source@snap2': unsupported version or feature # zfs send -L -i testpool/source@snap1 testpool/source@snap2 >/dev/null zfs-dbgmsg will provide the details of course: 2017 Sep 13 14:51:48: do_dump(): Aborting send as it would require splitting \ of large blocks which can cause silent data corruption. Set vfs.zfs.risk_data_corruption=1 \ to allow this or send with -L flag. See also: https://lists.freebsd.org/pipermail/freebsd-fs/2017-August/025110.html https://github.com/zfsonlinux/zfs/issues/6224 https://www.fabiankeil.de/tmp/reproduce-zfsonlinux-bug-6224.sh Obtained from: ElectroBSD --- .../opensolaris/uts/common/fs/zfs/dmu_send.c | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c index 47bb3f6428b5..c6e10c693cb1 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c @@ -73,6 +73,9 @@ int zfs_recv_queue_length = 16 * 1024 * 1024; /* Set this tunable to FALSE to disable setting of DRR_FLAG_FREERECORDS */ int zfs_send_set_freerecords_bit = B_TRUE; +/* Set this tunable to 1 to enable code paths that are known to cause data corruption */ +int zfs_risk_data_corruption = 0; + #ifdef _KERNEL SYSCTL_DECL(_vfs_zfs); SYSCTL_INT(_vfs_zfs, OID_AUTO, send_corrupt_data, CTLFLAG_RWTUN, @@ -80,6 +83,9 @@ SYSCTL_INT(_vfs_zfs, OID_AUTO, send_corrupt_data, CTLFLAG_RWTUN, "Blocks filled with 0x'zfs badd bloc' are send as replacement blocks."); SYSCTL_INT(_vfs_zfs, OID_AUTO, send_set_freerecords_bit, CTLFLAG_RWTUN, &zfs_send_set_freerecords_bit, 0, "Set the freerecords bits when sending"); +SYSCTL_INT(_vfs_zfs, OID_AUTO, risk_data_corruption, CTLFLAG_RWTUN, + &zfs_risk_data_corruption, 0, + "Enable code paths that are known to lead to data corruption under the right circumstances"); #endif static char *dmu_recv_tag = "dmu_recv_tag"; @@ -707,6 +713,25 @@ do_dump(dmu_sendarg_t *dsa, struct send_block_record *data) */ boolean_t split_large_blocks = blksz > SPA_OLD_MAXBLOCKSIZE && !(dsa->dsa_featureflags & DMU_BACKUP_FEATURE_LARGE_BLOCKS); + + /* + * Splitting large blocks can cause silent data corruption + * so we only "support" this if the user wants to risk it. + */ + if (split_large_blocks && !zfs_risk_data_corruption) { + static const char msg[] = "Aborting send as it would " + "require splitting of large blocks which can " + "cause silent data corruption. Set " + "vfs.zfs.risk_data_corruption=1 to allow this or" + "send with -L flag (if the receiver supports it)."; +#ifdef _KERNEL + zfs_dbgmsg("%s: %s", __func__, msg); +#else + fprintf(stderr, "%s: %s\n", __func__, msg); +#endif + return (SET_ERROR(ENOTSUP)); + } + /* * We should only request compressed data from the ARC if all * the following are true: -- 2.32.0 From 4588a53ae90bdbfe609caddb05228b79de9bd455 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 12 Nov 2021 20:57:48 +0100 Subject: [PATCH 285/325] ELECTRO_BLOAT i386: Sync with GENERIC Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BLOAT | 53 ++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 27 deletions(-) diff --git a/sys/i386/conf/ELECTRO_BLOAT b/sys/i386/conf/ELECTRO_BLOAT index fcfb7c314411..9f9df8483aaa 100644 --- a/sys/i386/conf/ELECTRO_BLOAT +++ b/sys/i386/conf/ELECTRO_BLOAT @@ -12,6 +12,7 @@ makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption +options VIMAGE # Subsystem virtualization, e.g. VNET options INET # InterNETworking options INET6 # IPv6 communications protocols options TCP_OFFLOAD # TCP offload @@ -32,8 +33,8 @@ options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. options GEOM_LABEL # Provides labelization -options COMPAT_FREEBSD9 # Compatible with FreeBSD9 options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options COMPAT_FREEBSD11 # Compatible with FreeBSD11 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options STACK # stack(9) support @@ -64,9 +65,16 @@ options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones +# Kernel dump features. +options EKCD # Support for encrypted kernel dumps +options GZIO # gzip-compressed kernel and user dumps +options ZSTDIO # zstd-compressed kernel and user dumps +options NETDUMP # netdump(4) client support + # To make an SMP kernel, the next two lines are needed options SMP # Symmetric MultiProcessor Kernel device apic # I/O APIC +options EARLY_AP_STARTUP # CPU frequency control device cpufreq @@ -74,22 +82,18 @@ device cpufreq # Bus support. device acpi device pci +options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers -options ATA_STATIC_ID # Static device numbering device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA # SCSI Controllers device ahc # AHA2940 and onboard AIC7xxx devices -options AHC_REG_PRETTY_PRINT # Print register bitfields in debug - # output. Adds ~128k to driver. -device ahd # AHA39320/29320 and onboard AIC79xx devices -options AHD_REG_PRETTY_PRINT # Print register bitfields in debug - # output. Adds ~215k to driver. + device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series device isp # Qlogic family @@ -98,15 +102,6 @@ device mps # LSI-Logic MPT-Fusion 2 device mpr # LSI-Logic MPT-Fusion 3 device trm # Tekram DC395U/UW/F DC315U adapters -device adv # Advansys SCSI adapters -device adw # Advansys wide SCSI adapters -device aha # Adaptec 154x SCSI adapters -device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. -device bt # Buslogic/Mylex MultiMaster SCSI adapters - -device ncv # NCR 53C500 -device nsp # Workbit Ninja SCSI-3 -device stg # TMC 18C30/18C50 device isci # Intel C600 SAS controller # ATA/SCSI peripherals @@ -140,6 +135,10 @@ device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s device pst # Promise Supertrak SX6000 device twe # 3ware ATA RAID +# NVM Express (NVMe) support +device nvme # base NVMe driver +device nvd # expose NVMe namespace as disks, depends on nvme + # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard @@ -162,9 +161,6 @@ device vt_vga device agp # support several AGP chipsets -# Add suspend/resume support for the i8254. -device pmtimer - # PCCARD (PCMCIA) support # PCMCIA and cardbus bridge support device cbb # cardbus (yenta) bridge @@ -183,11 +179,15 @@ device ppi # Parallel port interface device device puc # Multi I/O cards and multi-channel UARTs +# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure +device iflib +device igc # Intel I225 2.5G Ethernet +device em # Intel PRO/1000 Gigabit Ethernet Family +device vmx # VMware VMXNET3 Ethernet + # PCI Ethernet NICs. device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel PRO/1000 Gigabit Ethernet Family -device igb # Intel PRO/1000 PCIE Server Gigabit Family -device ixgb # Intel PRO/10GbE Ethernet Card device le # AMD Am7900 LANCE and Am79C9xx PCnet device ti # Alteon Networks Tigon I/II gigabit Ethernet device txp # 3Com 3cR990 (``Typhoon'') @@ -237,7 +237,6 @@ device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards device ex # Intel EtherExpress Pro/10 and Pro/10+ device ep # Etherlink III based cards device fe # Fujitsu MB8696x based cards -device ie # EtherExpress 8/16, 3C507, StarLAN 10 etc. device sn # SMC's 9000 series of Ethernet chips device xe # Xircom pccard Ethernet @@ -269,7 +268,7 @@ device padlock_rng # VIA Padlock RNG device rdrand_rng # Intel Bull Mountain RNG device ether # Ethernet support device vlan # 802.1Q VLAN support -device tun # Packet tunnel. +device tuntap # Packet tunnel. device md # Memory "disks" device gif # IPv6 and IPv4 tunneling device firmware # firmware assist module @@ -311,7 +310,10 @@ device virtio_blk # VirtIO Block device device virtio_scsi # VirtIO SCSI device device virtio_balloon # VirtIO Memory Balloon device -# HyperV drivers and enchancement support +# Linux KVM paravirtualization support +device kvm_clock # KVM paravirtual clock driver + +# HyperV drivers and enhancement support device hyperv # HyperV drivers # Xen HVM Guest Optimizations @@ -319,9 +321,6 @@ device hyperv # HyperV drivers options XENHVM # Xen HVM kernel infrastructure device xenpci # Xen HVM Hypervisor services driver -# VMware support -device vmx # VMware VMXNET3 Ethernet - # The crypto framework is required by IPSEC device crypto # Required by IPSEC -- 2.32.0 From b551f3ccb68916d2efeaf77dec98064e0510e71a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 13 Nov 2021 18:02:15 +0100 Subject: [PATCH 286/325] ELECTRO_BLOAT i386: Ditch "options KVA_PAGES=512" It seems to break the build and it looks like it's no longer needed anyway. [fk@elektrobier3 ~]$ time BUILD_JAIL=ElectroBSD-i386 sh /usr/src/reproduce-electrobsd.sh [...] cc -target i386-unknown-freebsd12.3 --sysroot=/usr/obj/usr/src/i386.i386/tmp -B/usr/obj/usr/src/i386.i386/tmp/usr/bin -c -x assembler-with-cpp -DLOCORE -O -pipe -g -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/ck/include -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -MD -MF.depend.locore.o -MTlocore.o -fdebug-prefix-map=./machine=/usr/src/sys/i386/include -fdebug-prefix-map=./x86=/usr/src/sys/x86/include -mno-mmx -mno-sse -msoft-float -ffreestanding -fwrapv -fstack-protector -gdwarf-2 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error=tautological-compare -Wno-error=empty-body -Wno-error=parentheses-equality -Wno-error=unused-function -Wno-error=pointer-sign -Wno-error=shift-negative-value -Wno-address-of-packed-member -Wno-format-zero-length -mno-aes -mno-avx -std=iso9899:1999 -Werror /usr/src/sys/i386/i386/locore.s In file included from /usr/src/sys/i386/i386/locore.s:51: ./machine/pmap.h:118:9: error: 'KVA_PAGES' macro redefined [-Werror,-Wmacro-redefined] #define KVA_PAGES (256*4) ^ ./opt_global.h:9:9: note: previous definition is here #define KVA_PAGES 512 ^ 1 error generated. *** Error code 1 Stop. make[2]: stopped in /usr/obj/usr/src/i386.i386/sys/ELECTRO_BLOAT *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src 31.22s real 11.87s user 6.65s sys 43148 maximum resident set size 1865 average shared memory size 54 average unshared data size 128 average unshared stack size 417404 page reclaims 1163 page faults 0 swaps 53338 block input operations 126 block output operations 0 messages sent 0 messages received 0 signals received 12584 voluntary context switches 46442 involuntary context switches ElectroBSD-i386: removed Script done, output file is /var/log/buildlogs/reprolog-2021-11-12_20:58 Script started, output file is /var/log/buildlogs/reprolog-2021-11-12_20:58 Script done, output file is /var/log/buildlogs/reprolog-2021-11-12_20:58 real 641m19.404s user 712m47.369s sys 24m25.377s Obtained from: ElectroBSD --- sys/i386/conf/ELECTRO_BLOAT | 4 ---- 1 file changed, 4 deletions(-) diff --git a/sys/i386/conf/ELECTRO_BLOAT b/sys/i386/conf/ELECTRO_BLOAT index 9f9df8483aaa..efeb7393f394 100644 --- a/sys/i386/conf/ELECTRO_BLOAT +++ b/sys/i386/conf/ELECTRO_BLOAT @@ -328,7 +328,3 @@ device crypto # Required by IPSEC # For details see UPDATING entry 20121223. After r286288 it's # probably no longer necessary, but for now we keep it anyway. options KSTACK_PAGES=4 - -# Increase the size of the kernel virtual address space -# so ZFS can cache more stuff. -options KVA_PAGES=512 -- 2.32.0 From aef30214dc985dbf527f5bc200d7b7de90c8b7c2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 13:27:28 +0100 Subject: [PATCH 287/325] ggated: Ignore SIGPIPE to prevent DoS ... by a single prematurely closed client connection. Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index f61efdc61690..afe16b625884 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1040,6 +1040,7 @@ main(int argc, char *argv[]) pidfile_write(pfh); signal(SIGCHLD, SIG_IGN); + signal(SIGPIPE, SIG_IGN); sfd = socket(AF_INET, SOCK_STREAM, 0); if (sfd == -1) -- 2.32.0 From 8c4e92932e54481c82985f0550bc0ccfec88c345 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 13:47:31 +0100 Subject: [PATCH 288/325] ggated: Remove connection if the initial packet couldn't be sent Should help to mitigate DoS after flooding ggated with incomplete requests: error: accept(): Software caused connection abort. error: Exiting. Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index afe16b625884..595845034993 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -943,6 +943,7 @@ handshake(struct sockaddr *from, int sfd) if (data == -1) { sendfail(sfd, errno, "Error while sending initial packet: %s.", strerror(errno)); + connection_remove(conn); return (0); } -- 2.32.0 From 9f8559ba7b8972b035d15c3f3482b52e422fd631 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 14:09:24 +0100 Subject: [PATCH 289/325] ggated: Continue if accept() is interrupted or the remote connection is lost Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 595845034993..147d41cc49bd 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1065,9 +1065,11 @@ main(int argc, char *argv[]) for (;;) { fromlen = sizeof(from); tmpsfd = accept(sfd, &from, &fromlen); - if (tmpsfd == -1) + if (tmpsfd == -1) { + if (errno == EINTR || errno == ECONNABORTED) + continue; g_gate_xlog("accept(): %s.", strerror(errno)); - + } if (got_sighup) { got_sighup = 0; exports_get(); -- 2.32.0 From 6f40be793013649d2b9086d7b3aa52b57f7b02a8 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 14:16:47 +0100 Subject: [PATCH 290/325] ggated: Prevent c_diskfd leaks through connection_remove() Should help against DoS: [...] debug: Connection created [127.0.0.1, /tank/scratch/testfile]. debug: New connection created (token=2197914058). debug: exports[/tank/scratch/testfile2]: Path mismatch. debug: Sending initial packet. error: accept(): Too many open files. error: Exiting. Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 147d41cc49bd..ae5308e7af34 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -527,6 +527,8 @@ connection_remove(struct ggd_connection *conn) close(conn->c_sendfd); if (conn->c_recvfd != -1) close(conn->c_recvfd); + if (conn->c_diskfd != -1) + close(conn->c_diskfd); free(conn->c_path); free(conn); } -- 2.32.0 From 8d378dbe9aaf5fc0e0e8ad499637ef663f1a29e7 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 9 Dec 2014 15:52:39 +0100 Subject: [PATCH 291/325] ggated: Check for connection_add() failures properly Prevents a socket leak Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index ae5308e7af34..769f76e1d788 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -905,7 +905,7 @@ handshake(struct sockaddr *from, int sfd) */ g_gate_log(LOG_DEBUG, "Found existing connection (token=%lu).", (unsigned long)conn->c_token); - if (connection_add(conn, &cinit, from, sfd) == -1) { + if (connection_add(conn, &cinit, from, sfd) == EEXIST) { connection_remove(conn); return (0); } -- 2.32.0 From ef0b7dbd84be74a8a7282b0fbfaef2d316281cc3 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 8 Dec 2014 17:59:38 +0100 Subject: [PATCH 292/325] ggated: Do not leak stack data in sendfail() Reported to security-officer@FreeBSD.org on 2014-12-09. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 769f76e1d788..6005221052e2 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -606,6 +606,7 @@ sendfail(int sfd, int error, const char *fmt, ...) va_list ap; ssize_t data; + bzero(&sinit, sizeof(sinit)); sinit.gs_error = error; g_gate_swap2n_sinit(&sinit); data = g_gate_send(sfd, &sinit, sizeof(sinit), 0); -- 2.32.0 From e89ed1acfac225df3920abb41ea0e7e3f58094ab Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 2 Apr 2015 15:24:58 +0200 Subject: [PATCH 293/325] ggated recv_thread(): Do not queue incomplete WRITE requests Verifying that g_gate_recv()'s return code isn't -1 is insufficient as it's a thin wrapper arround recv(2) which, quoting its man page, "may still return less data than requested if a signal is caught, an error or disconnect occurs, or the next data to be received is of a different type than that returned". Previously incomplete WRITE requests would be scheduled with partially uninitialized memory, potentially resulting in file system corruption or, worse, bogus data being later on returned as valid. Security impact: A MITM may cause data corruption by disrupting the connection from ggatec's send_thread() to ggated's recv_thread() at the right point in time. This does not require access to the plain text traffic but if encryption is involved the attacker would have to guess that it's ggate traffic and disrupt connections blindly, hoping that some of the disruptions trigger the bug. The issue was discovered after ZFS on the ggatec side reported checksum errors which weren't reproducible on the ggated side where ZFS had received and checksummed bogus data. The ggate traffic was tunneled through SSH and Tor with sshd running as Tor location hidden service. Reported to security-officer@FreeBSD.org on 2015-04-05. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 6005221052e2..525d92bd4957 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -684,6 +684,9 @@ recv_thread(void *arg) if (data == -1) { g_gate_xlog("Error while receiving data: %s.", strerror(errno)); + } else if ((uint32_t)data != req->r_length) { + g_gate_xlog("Received %d bytes of data while " + "expecting %u.", data, req->r_length); } } -- 2.32.0 From d705fbd89485143d85a16d2f51c373414ae54df2 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 2 Apr 2015 12:09:40 +0200 Subject: [PATCH 294/325] ggated recv_thread(): Do not queue requests with invalid values ... that would cause abort()s when read by the disk_thread() later on. From ggatec's point of view it doesn't make a difference as the connection will get closed either way, but at least the admin on the server side doesn't have to deal with core dumps. Security impact: An authenticated attacker may intentionally cause the ggated process that handles the attacker's connection to core dump and thus use more disk space than intentionally provisioned by the server admin. Without the following patch ggated core dumps may require more than 100 GB of disk space. Reported to security-officer@FreeBSD.org on 2015-04-05. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 525d92bd4957..29c227dbd693 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -668,6 +668,23 @@ recv_thread(void *arg) g_gate_log(LOG_DEBUG, "%s: offset=%" PRIu64 " length=%" PRIu32, __func__, req->r_offset, req->r_length); + /* + * Reject requests that violate assertions in disk_thread(). + */ + if (req->r_cmd != GGATE_CMD_READ && + req->r_cmd != GGATE_CMD_WRITE) { + g_gate_xlog("Request contains invalid command."); + } + if (req->r_offset + req->r_length > + (uintmax_t)conn->c_mediasize) { + g_gate_xlog("Request out of bounds."); + } + if (req->r_offset % conn->c_sectorsize != 0 || + req->r_length % conn->c_sectorsize != 0) { + g_gate_xlog("Request length or offset does " + "not fit sector size."); + } + /* * Allocate memory for data. */ -- 2.32.0 From 50d6e3287605abbf7f624fd7f19357b2f27b7091 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 2 Apr 2015 19:52:54 +0200 Subject: [PATCH 295/325] ggated recv_thread(): Reject request with more than MAXPHYS bytes of data .. to limit the amount of memory we (try to) allocate on behalf of the client without knowing whether or not the client actually intents to use it. MAXPHYS is the hardcoded limit in ggatec so anything above it is suspicious and could be a DoS attempt. This commit forces users who like to tune MAXPHYS to make sure the value used by ggated is not below the one used by ggatec. While not ideal, this seems preferable to the DoS risk. Reported to security-officer@FreeBSD.org on 2015-04-05. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 29c227dbd693..edbf9b952ba7 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -685,6 +685,16 @@ recv_thread(void *arg) "not fit sector size."); } + /* + * Limit the amount of memory we allocate on behalf of + * the client. MAXPHYS is the hard limit in ggatec, + * values above it are thus pretty suspicious. + */ + if (req->r_length > MAXPHYS) { + g_gate_xlog("Request length above MAXPHYS: %u > %u", + (unsigned)req->r_length, MAXPHYS); + } + /* * Allocate memory for data. */ -- 2.32.0 From 298f1785a64220dd811417b0332a4ae25d91e7dd Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 24 Apr 2015 14:04:31 +0200 Subject: [PATCH 296/325] ggatec: Add support for SOCKS5 with domain names Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 104 +++++++++++++++++++++++++++++++++++-- 1 file changed, 99 insertions(+), 5 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 0695dae0dca2..c647deb614b5 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -67,6 +67,8 @@ static unsigned flags = 0; static int force = 0; static unsigned queue_size = G_GATE_QUEUE_SIZE; static unsigned port = G_GATE_PORT; +static char *socks_dest = NULL; +static unsigned dest_port = 3080; static off_t mediasize; static unsigned sectorsize = 0; static unsigned timeout = G_GATE_TIMEOUT; @@ -81,9 +83,11 @@ usage(void) fprintf(stderr, "usage: %s create [-nv] [-o ] [-p port] " "[-q queue_size] [-R rcvbuf] [-S sndbuf] [-s sectorsize] " - "[-t timeout] [-u unit] \n", getprogname()); + "[-t timeout] [-T :] [-u unit] \n", + getprogname()); fprintf(stderr, " %s rescue [-nv] [-o ] [-p port] " - "[-R rcvbuf] [-S sndbuf] <-u unit> \n", getprogname()); + "[-R rcvbuf] [-S sndbuf] [-T :] <-u unit> " + " \n", getprogname()); fprintf(stderr, " %s destroy [-f] <-u unit>\n", getprogname()); fprintf(stderr, " %s list [-v] [-u unit]\n", getprogname()); exit(EXIT_FAILURE); @@ -262,6 +266,69 @@ recv_thread(void *arg __unused) pthread_exit(NULL); } +static void +negotiate_socks_connection(int sfd) +{ + struct negotiation_request { + char version; + char nmethods; + char method; + } neg_request; + struct socks_request { + char version; + char cmd; + char reserved; + char address_type; + char host_length; + char dest[255 + 2]; + } socks_request; + char response[10]; + size_t request_length; + size_t host_length; + + host_length = strlen(socks_dest); + + neg_request.version = '\x05'; + neg_request.nmethods = '\x01'; /* We support one method: */ + neg_request.method = '\x00'; /* no authentication */ + + g_gate_log(LOG_DEBUG, "Starting SOCKS negotiation."); + if (g_gate_send(sfd, &neg_request, sizeof(neg_request), MSG_NOSIGNAL) == -1) + g_gate_xlog("Failed to send SOCKS negotiation request."); + + if (g_gate_recv(sfd, &response, sizeof(response), MSG_WAITALL) != 2) + g_gate_xlog("Failed to read SOCKS negotiation response."); + + if (response[0] != '\x05' || response[1] != '\x00') + g_gate_xlog("SOCKS negotiation failed."); + + g_gate_log(LOG_DEBUG, "Negotiated SOCKS5. " + "Requesting connection to %s:%d.", socks_dest, dest_port); + + socks_request.version = '\x05'; + socks_request.cmd = '\x01'; /* Connect */ + socks_request.reserved = '\x00'; + socks_request.address_type = '\x03'; /* Address is domain name */; + socks_request.host_length = (char)host_length; + strncpy(socks_request.dest, socks_dest, host_length); + socks_request.dest[host_length] = (char)((dest_port >> 8) & 0xff); + socks_request.dest[host_length + 1] = (char)(dest_port & 0xff); + request_length = sizeof(socks_request) - sizeof(socks_request.dest) + + host_length + 2; + + if (g_gate_send(sfd, &socks_request, request_length, MSG_NOSIGNAL) == -1) + g_gate_xlog("Failed to send SOCKS5 request."); + + if (g_gate_recv(sfd, &response, sizeof(response), MSG_WAITALL) != sizeof(response)) + g_gate_xlog("Failed to read SOCKS5 response."); + + if (response[0] != '\x05' || response[1] != '\x00') + g_gate_xlog("Failed to SOCKS5 connect to %s:%d", + socks_dest, dest_port); + + g_gate_log(LOG_INFO, "Connected to: %s:%d.", socks_dest, dest_port); +} + static int handshake(int dir) { @@ -300,6 +367,9 @@ handshake(int dir) g_gate_log(LOG_INFO, "Connected to the server: %s:%d.", host, port); + if (socks_dest != NULL) + negotiate_socks_connection(sfd); + /* * Create and send version packet. */ @@ -479,8 +549,13 @@ g_gatec_create(void) ggioc.gctl_maxcount = queue_size; ggioc.gctl_timeout = timeout; ggioc.gctl_unit = unit; - snprintf(ggioc.gctl_info, sizeof(ggioc.gctl_info), "%s:%u %s", host, - port, path); + if (socks_dest != NULL) + snprintf(ggioc.gctl_info, sizeof(ggioc.gctl_info), + "socks5://%s:%u -> %s:%u %s", host, + port, socks_dest, dest_port, path); + else + snprintf(ggioc.gctl_info, sizeof(ggioc.gctl_info), "%s:%u %s", + host, port, path); g_gate_ioctl(G_GATE_CMD_CREATE, &ggioc); if (unit == -1) { printf("%s%u\n", G_GATE_PROVIDER_NAME, ggioc.gctl_unit); @@ -529,8 +604,9 @@ main(int argc, char *argv[]) argv += 1; for (;;) { int ch; + char *p; - ch = getopt(argc, argv, "fno:p:q:R:S:s:t:u:v"); + ch = getopt(argc, argv, "fno:p:q:R:S:s:t:T:u:v"); if (ch == -1) break; switch (ch) { @@ -598,6 +674,24 @@ main(int argc, char *argv[]) if (sectorsize == 0 && errno != 0) errx(EXIT_FAILURE, "Invalid sectorsize."); break; + case 'T': + if (action != CREATE && action != RESCUE) + usage(); + socks_dest = optarg; + p = strchr(socks_dest, ':'); + if (p != NULL) { + errno = 0; + *p = '\0'; + p++; + dest_port = strtoul(p, NULL, 10); + if (dest_port == 0 && errno != 0) + errx(EXIT_FAILURE, + "Invalid socks5t port: %s.", p); + } + if (strlen(socks_dest) > (size_t)255) + errx(EXIT_FAILURE, + "Socks destination address too long."); + break; case 't': if (action != CREATE) usage(); -- 2.32.0 From 7c7cd8f462872b0d4f03d0e7a04b3173fa2fb79e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 24 Apr 2015 15:26:42 +0200 Subject: [PATCH 297/325] ggatec.8: Document SOCKS5 support Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.8 | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/sbin/ggate/ggatec/ggatec.8 b/sbin/ggate/ggatec/ggatec.8 index 6f761dcfd99b..ba3c9ecb6f0b 100644 --- a/sbin/ggate/ggatec/ggatec.8 +++ b/sbin/ggate/ggatec/ggatec.8 @@ -41,6 +41,7 @@ .Op Fl R Ar rcvbuf .Op Fl S Ar sndbuf .Op Fl s Ar sectorsize +.Op Fl T Ar remote_target:port .Op Fl t Ar timeout .Op Fl u Ar unit .Ar host @@ -53,6 +54,7 @@ .Op Fl p Ar port .Op Fl R Ar rcvbuf .Op Fl S Ar sndbuf +.Op Fl T Ar remote_target:port .Fl u Ar unit .Ar host .Ar path @@ -137,6 +139,9 @@ Sector size for .Nm ggate provider. If not specified, it is taken from the device, or set to 512 bytes for files. +.It Fl T Ar remote_host:port +Use SOCK5 to open connection to remote_host:port before switching +to the ggated protocol. .It Fl t Ar timeout Number of seconds to wait before an I/O request will be canceled. Default is 0, which means no timeout. @@ -167,6 +172,14 @@ server# ggated client# ggatec create -o ro server /dev/cd0 ggate0 client# mount_cd9660 /dev/ggate0 /cdrom + +.Ed +Connect to 127.0.1.1:9050, SOCKS5-negotiate a connection to +the Tor location hidden service czdqtfrgvizltdal.onion:1312 +and access a ZVOL: +.Bd -literal -offset indent +# ggatec create -T czdqtfrgvizltdal.onion:1312 -p 9050 \\ + 127.0.1.1 /dev/zvol/dpool/ggated/czdqtfrgvizltdal.eli .Ed .Sh SEE ALSO .Xr geom 4 , -- 2.32.0 From 549715b0683576ab970b4a2a9a9643155321e152 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 27 Apr 2015 19:10:17 +0200 Subject: [PATCH 298/325] ggatec: Reject unexpected GGATE commands in recv_thread() Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index c647deb614b5..99e46697efbe 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -242,6 +242,10 @@ recv_thread(void *arg __unused) g_gate_log(LOG_ERR, "Received too big response: %zd", ggio.gctl_length); break; } + if (ggio.gctl_cmd != GGATE_CMD_READ && + ggio.gctl_cmd != GGATE_CMD_WRITE) { + g_gate_xlog("Unexpected GGATE_CMD: %d", ggio.gctl_cmd); + } if (ggio.gctl_error == 0 && ggio.gctl_cmd == GGATE_CMD_READ) { data = g_gate_recv(recvfd, ggio.gctl_data, -- 2.32.0 From 709bea3381db17eb20f90bb8055665cc5e685515 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 27 Apr 2015 19:15:18 +0200 Subject: [PATCH 299/325] ggatec: Log if the remote side signals errors Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 99e46697efbe..171c25b7d781 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -247,6 +247,12 @@ recv_thread(void *arg __unused) g_gate_xlog("Unexpected GGATE_CMD: %d", ggio.gctl_cmd); } + if (ggio.gctl_error != 0) { + g_gate_log(LOG_ERR, + "Remote side signaled error %d: %s.", + ggio.gctl_error, strerror(ggio.gctl_error)); + } + if (ggio.gctl_error == 0 && ggio.gctl_cmd == GGATE_CMD_READ) { data = g_gate_recv(recvfd, ggio.gctl_data, ggio.gctl_length, MSG_WAITALL); -- 2.32.0 From c283751db430acac3827e1bd1dadda0f63da1247 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 27 Apr 2015 19:53:30 +0200 Subject: [PATCH 300/325] ggate[cd]: Add BIO_FLUSH support Let ggated transform BIO_FLUSH requests into fsync() calls. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 8 +++++++- sbin/ggate/ggated/ggated.c | 23 ++++++++++++++++++----- sbin/ggate/shared/ggate.h | 1 + 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 171c25b7d781..b9f159ce272f 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -149,6 +149,11 @@ send_thread(void *arg __unused) case BIO_WRITE: hdr.gh_cmd = GGATE_CMD_WRITE; break; + case BIO_FLUSH: + g_gate_log(LOG_DEBUG, "FLUSH request"); + hdr.gh_cmd = GGATE_CMD_FLUSH; + assert(ggio.gctl_length == 0); + break; default: g_gate_log(LOG_NOTICE, "Unknown gctl_cmd: %i", ggio.gctl_cmd); ggio.gctl_error = EOPNOTSUPP; @@ -243,7 +248,8 @@ recv_thread(void *arg __unused) break; } if (ggio.gctl_cmd != GGATE_CMD_READ && - ggio.gctl_cmd != GGATE_CMD_WRITE) { + ggio.gctl_cmd != GGATE_CMD_WRITE && + ggio.gctl_cmd != GGATE_CMD_FLUSH) { g_gate_xlog("Unexpected GGATE_CMD: %d", ggio.gctl_cmd); } diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index edbf9b952ba7..12517a51d70f 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -672,8 +672,10 @@ recv_thread(void *arg) * Reject requests that violate assertions in disk_thread(). */ if (req->r_cmd != GGATE_CMD_READ && - req->r_cmd != GGATE_CMD_WRITE) { - g_gate_xlog("Request contains invalid command."); + req->r_cmd != GGATE_CMD_WRITE && + req->r_cmd != GGATE_CMD_FLUSH) { + g_gate_xlog("Request contains invalid command: %d", + req->r_cmd); } if (req->r_offset + req->r_length > (uintmax_t)conn->c_mediasize) { @@ -696,9 +698,10 @@ recv_thread(void *arg) } /* - * Allocate memory for data. + * Allocate memory for data, except when flushing. */ - req->r_data = malloc_waitok(req->r_length); + req->r_data = req->r_cmd != GGATE_CMD_FLUSH ? + malloc_waitok(req->r_length) : NULL; /* * Receive data to write for WRITE request. @@ -758,7 +761,9 @@ disk_thread(void *arg) /* * Check the request. */ - assert(req->r_cmd == GGATE_CMD_READ || req->r_cmd == GGATE_CMD_WRITE); + assert(req->r_cmd == GGATE_CMD_READ || + req->r_cmd == GGATE_CMD_WRITE || + req->r_cmd == GGATE_CMD_FLUSH); assert(req->r_offset + req->r_length <= (uintmax_t)conn->c_mediasize); assert((req->r_offset % conn->c_sectorsize) == 0); assert((req->r_length % conn->c_sectorsize) == 0); @@ -782,6 +787,14 @@ disk_thread(void *arg) free(req->r_data); req->r_data = NULL; break; + case GGATE_CMD_FLUSH: + g_gate_log(LOG_DEBUG, "Flushing"); + if (fsync(fd)) { + req->r_error = errno; + g_gate_log(LOG_ERR, "Flushing failed: %s", + strerror(errno)); + } + break; } if (data != (ssize_t)req->r_length) { /* Report short reads/writes as I/O errors. */ diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 9ea901036726..a649e7fd322d 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -57,6 +57,7 @@ #define GGATE_CMD_READ 0 #define GGATE_CMD_WRITE 1 +#define GGATE_CMD_FLUSH 2 extern int g_gate_devfd; extern int g_gate_verbose; -- 2.32.0 From 7d610ef06f4000a718f71282571e3a3dc9025145 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 29 Apr 2015 12:44:56 +0200 Subject: [PATCH 301/325] ggatec: Log the command type for hdr packets (when debugging) ... and provide more details about failed requests. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 11 ++++++++--- sbin/ggate/shared/ggate.c | 16 ++++++++++++++++ sbin/ggate/shared/ggate.h | 1 + 3 files changed, 25 insertions(+), 3 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index b9f159ce272f..3be5d63a1ac9 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -176,7 +176,9 @@ send_thread(void *arg __unused) g_gate_swap2n_hdr(&hdr); data = g_gate_send(sendfd, &hdr, sizeof(hdr), MSG_NOSIGNAL); - g_gate_log(LOG_DEBUG, "Sent hdr packet."); + g_gate_log(LOG_DEBUG, "Sent hdr packet (%s).", + g_gate_cmd2str(hdr.gh_cmd)); + g_gate_swap2h_hdr(&hdr); if (reconnect) break; @@ -234,7 +236,8 @@ recv_thread(void *arg __unused) pthread_kill(sendtd, SIGUSR1); break; } - g_gate_log(LOG_DEBUG, "Received hdr packet."); + g_gate_log(LOG_DEBUG, "Received hdr packet (%s).", + g_gate_cmd2str(hdr.gh_cmd)); ggio.gctl_seq = hdr.gh_seq; ggio.gctl_cmd = hdr.gh_cmd; @@ -255,7 +258,9 @@ recv_thread(void *arg __unused) if (ggio.gctl_error != 0) { g_gate_log(LOG_ERR, - "Remote side signaled error %d: %s.", + "%s for %d bytes at offset %d failed. " + "Error %d: %s.", g_gate_cmd2str(ggio.gctl_cmd), + ggio.gctl_length, ggio.gctl_offset, ggio.gctl_error, strerror(ggio.gctl_error)); } diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index 2f544f7a2c9c..6197540d9678 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -409,3 +409,19 @@ g_gate_str2ip(const char *str) return (INADDR_NONE); return (((struct in_addr *)(void *)hp->h_addr)->s_addr); } + +const char * +g_gate_cmd2str(int cmd) +{ + + switch (cmd) { + case GGATE_CMD_READ: + return ("GGATE_CMD_READ"); + case GGATE_CMD_WRITE: + return ("GGATE_CMD_WRITE"); + case GGATE_CMD_FLUSH: + return ("GGATE_CMD_FLUSH"); + } + + return ("unknown (invalid?) GGATE command"); +} diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index a649e7fd322d..50758708f522 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -114,6 +114,7 @@ void g_gate_socket_settings(int sfd); void g_gate_list(int unit, int verbose); #endif in_addr_t g_gate_str2ip(const char *str); +const char *g_gate_cmd2str(int cmd); /* * g_gate_swap2h_* - functions swap bytes to host byte order (from big endian). -- 2.32.0 From e5d139425b3340ecbc872ac148397a22d2a65f99 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 6 May 2015 15:55:08 +0200 Subject: [PATCH 302/325] ggated disk_thread(): Include the command in the debug output Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 12517a51d70f..5bc3979506c7 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -768,8 +768,10 @@ disk_thread(void *arg) assert((req->r_offset % conn->c_sectorsize) == 0); assert((req->r_length % conn->c_sectorsize) == 0); - g_gate_log(LOG_DEBUG, "%s: offset=%" PRIu64 " length=%" PRIu32, - __func__, req->r_offset, req->r_length); + g_gate_log(LOG_DEBUG, + "%s: cmd=%s offset=%" PRIu64 " length=%" PRIu32, + __func__, g_gate_cmd2str(req->r_cmd), req->r_offset, + req->r_length); /* * Do the request. -- 2.32.0 From 921592c6693e834f7162c1772330dfe675e3f322 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 5 May 2015 17:39:16 +0200 Subject: [PATCH 303/325] ggate[cd]: Add BIO_DELETE support On the ggated side the requests are translated into writes of zero which ZFS will convert into BIO_DELETE requests again when zle compression is enabled. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 5 ++++ sbin/ggate/ggated/ggated.c | 47 ++++++++++++++++++++++++++++++++------ sbin/ggate/shared/ggate.c | 2 ++ sbin/ggate/shared/ggate.h | 1 + 4 files changed, 48 insertions(+), 7 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 3be5d63a1ac9..b60fb35d5ff5 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -149,6 +149,10 @@ send_thread(void *arg __unused) case BIO_WRITE: hdr.gh_cmd = GGATE_CMD_WRITE; break; + case BIO_DELETE: + g_gate_log(LOG_DEBUG, "DELETE request"); + hdr.gh_cmd = GGATE_CMD_DELETE; + break; case BIO_FLUSH: g_gate_log(LOG_DEBUG, "FLUSH request"); hdr.gh_cmd = GGATE_CMD_FLUSH; @@ -252,6 +256,7 @@ recv_thread(void *arg __unused) } if (ggio.gctl_cmd != GGATE_CMD_READ && ggio.gctl_cmd != GGATE_CMD_WRITE && + ggio.gctl_cmd != GGATE_CMD_DELETE && ggio.gctl_cmd != GGATE_CMD_FLUSH) { g_gate_xlog("Unexpected GGATE_CMD: %d", ggio.gctl_cmd); } diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 5bc3979506c7..8bb5972ac576 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -651,6 +651,7 @@ recv_thread(void *arg) * Get header packet. */ req = malloc_waitok(sizeof(*req)); + memset(req, 0, sizeof(*req)); data = g_gate_recv(fd, &req->r_hdr, sizeof(req->r_hdr), MSG_WAITALL); if (data == 0) { @@ -673,6 +674,7 @@ recv_thread(void *arg) */ if (req->r_cmd != GGATE_CMD_READ && req->r_cmd != GGATE_CMD_WRITE && + req->r_cmd != GGATE_CMD_DELETE && req->r_cmd != GGATE_CMD_FLUSH) { g_gate_xlog("Request contains invalid command: %d", req->r_cmd); @@ -692,21 +694,16 @@ recv_thread(void *arg) * the client. MAXPHYS is the hard limit in ggatec, * values above it are thus pretty suspicious. */ - if (req->r_length > MAXPHYS) { + if (req->r_length > MAXPHYS && req->r_cmd != GGATE_CMD_DELETE) { g_gate_xlog("Request length above MAXPHYS: %u > %u", (unsigned)req->r_length, MAXPHYS); } - /* - * Allocate memory for data, except when flushing. - */ - req->r_data = req->r_cmd != GGATE_CMD_FLUSH ? - malloc_waitok(req->r_length) : NULL; - /* * Receive data to write for WRITE request. */ if (req->r_cmd == GGATE_CMD_WRITE) { + req->r_data = malloc_waitok(req->r_length); g_gate_log(LOG_DEBUG, "Waiting for %u bytes of data...", req->r_length); data = g_gate_recv(fd, req->r_data, req->r_length, @@ -733,6 +730,34 @@ recv_thread(void *arg) } } +static ssize_t +delete_range(int fd, size_t length, off_t offset) +{ + static char zeros[MAXPHYS]; + size_t written; + + written = 0; + + do + { + int ret; + size_t bytes_left; + size_t chunk_size; + + bytes_left = length - written; + chunk_size = bytes_left > MAXPHYS ? MAXPHYS : bytes_left; + ret = pwrite(fd, zeros, chunk_size, offset + written); + if (ret == -1) + return (written); + written += ret; + } while (written < length); + + g_gate_log(LOG_DEBUG, "Overwritten %u bytes at offset %jd with zeros", + written, (intmax_t)offset); + + return (written); +} + static void * disk_thread(void *arg) { @@ -763,6 +788,7 @@ disk_thread(void *arg) */ assert(req->r_cmd == GGATE_CMD_READ || req->r_cmd == GGATE_CMD_WRITE || + req->r_cmd == GGATE_CMD_DELETE || req->r_cmd == GGATE_CMD_FLUSH); assert(req->r_offset + req->r_length <= (uintmax_t)conn->c_mediasize); assert((req->r_offset % conn->c_sectorsize) == 0); @@ -779,9 +805,16 @@ disk_thread(void *arg) data = 0; switch (req->r_cmd) { case GGATE_CMD_READ: + assert(req->r_data == NULL); + req->r_data = malloc_waitok(req->r_length); data = pread(fd, req->r_data, req->r_length, req->r_offset); break; + case GGATE_CMD_DELETE: + data = delete_range(fd, req->r_length, + req->r_offset); + assert((size_t)data <= req->r_length); + break; case GGATE_CMD_WRITE: data = pwrite(fd, req->r_data, req->r_length, req->r_offset); diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index 6197540d9678..d6be4948d9c6 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -419,6 +419,8 @@ g_gate_cmd2str(int cmd) return ("GGATE_CMD_READ"); case GGATE_CMD_WRITE: return ("GGATE_CMD_WRITE"); + case GGATE_CMD_DELETE: + return ("GGATE_CMD_DELETE"); case GGATE_CMD_FLUSH: return ("GGATE_CMD_FLUSH"); } diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 50758708f522..cc67ca49571f 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -58,6 +58,7 @@ #define GGATE_CMD_READ 0 #define GGATE_CMD_WRITE 1 #define GGATE_CMD_FLUSH 2 +#define GGATE_CMD_DELETE 3 extern int g_gate_devfd; extern int g_gate_verbose; -- 2.32.0 From a979f1c8cb65fbf629122563af18382e64e82775 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 29 Apr 2015 10:55:40 +0200 Subject: [PATCH 304/325] ggated send_thread(): Assert that we only send data for read requests Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 8bb5972ac576..faf44cc11423 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -899,6 +899,7 @@ send_thread(void *arg) g_gate_log(LOG_DEBUG, "Sent hdr packet."); g_gate_swap2h_hdr(&req->r_hdr); if (req->r_data != NULL) { + assert(req->r_cmd == GGATE_CMD_READ); data = g_gate_send(fd, req->r_data, req->r_length, 0); if (data != (ssize_t)req->r_length) { g_gate_xlog("Error while sending data: %s.", -- 2.32.0 From 1bc179d722584b0c9a683c2dbe7e35d2c5174616 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 4 May 2015 18:00:04 +0200 Subject: [PATCH 305/325] ggated: Open the listening socket CLOEXEC Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index faf44cc11423..436f7f8a6d98 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1125,7 +1125,7 @@ main(int argc, char *argv[]) signal(SIGCHLD, SIG_IGN); signal(SIGPIPE, SIG_IGN); - sfd = socket(AF_INET, SOCK_STREAM, 0); + sfd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0); if (sfd == -1) g_gate_xlog("Cannot open stream socket: %s.", strerror(errno)); bzero(&serv, sizeof(serv)); -- 2.32.0 From fc6a781038ceb31871a44952171ee6825d42b45f Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 4 May 2015 18:31:46 +0200 Subject: [PATCH 306/325] ggated: Fix another socket leak Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 436f7f8a6d98..fed36964939f 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -344,6 +344,11 @@ exports_check(struct ggd_export *ex, struct g_gate_cinit *cinit, return (EPERM); } } + if (conn->c_diskfd != -1) { + g_gate_log(LOG_DEBUG, "Requested file %s is already open: %d", + ex->e_path, conn->c_diskfd); + return(0); + } if ((conn->c_flags & GGATE_FLAG_RDONLY) != 0) flags = O_RDONLY; else if ((conn->c_flags & GGATE_FLAG_WRONLY) != 0) -- 2.32.0 From bb7f64f0a38414e95f5abebfcd63da70c3111065 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 30 Apr 2015 11:52:06 +0200 Subject: [PATCH 307/325] ggated recv_thread(): In case of read-only files, only accept read commands Accepting write commands etc. is not a security problem because the file descriptor isn't writeable anyway, but accepting requests other than reads could hide client bugs. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index fed36964939f..f811b853f79c 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -694,6 +694,12 @@ recv_thread(void *arg) "not fit sector size."); } + if ((conn->c_flags & GGATE_FLAG_RDONLY) != 0 + && req->r_cmd != GGATE_CMD_READ) { + g_gate_xlog("%s request received for read-only file", + g_gate_cmd2str(req->r_cmd)); + } + /* * Limit the amount of memory we allocate on behalf of * the client. MAXPHYS is the hard limit in ggatec, -- 2.32.0 From c19877218eae18fa21b86d7d0acb08780de5eb04 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 3 May 2015 14:02:02 +0200 Subject: [PATCH 308/325] ggatec: Add log-to-file support Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 5 ++++- sbin/ggate/shared/ggate.c | 26 +++++++++++++++++++++----- sbin/ggate/shared/ggate.h | 1 + 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index b60fb35d5ff5..6af026e80277 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -632,7 +632,7 @@ main(int argc, char *argv[]) int ch; char *p; - ch = getopt(argc, argv, "fno:p:q:R:S:s:t:T:u:v"); + ch = getopt(argc, argv, "fl:no:p:q:R:S:s:t:T:u:v"); if (ch == -1) break; switch (ch) { @@ -641,6 +641,9 @@ main(int argc, char *argv[]) usage(); force = 1; break; + case 'l': + g_gate_open_log(optarg); + break; case 'n': if (action != CREATE && action != RESCUE) usage(); diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index d6be4948d9c6..084db7304e53 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -28,6 +28,7 @@ * $FreeBSD$ */ +#define _WITH_DPRINTF #include #include #include @@ -61,13 +62,23 @@ int g_gate_devfd = -1; int g_gate_verbose = 0; +static int g_gate_logfd = -1; +void +g_gate_open_log(const char *logfile) +{ + + g_gate_logfd = open(logfile, O_CREAT | O_WRONLY | O_APPEND, S_IWUSR |S_IRUSR); + if (g_gate_logfd == -1) { + g_gate_xlog("Failed to open %s: %s", logfile, strerror(errno)); + } +} void g_gate_vlog(int priority, const char *message, va_list ap) { - if (g_gate_verbose) { + if (g_gate_verbose || g_gate_logfd != -1) { const char *prefix; switch (priority) { @@ -89,10 +100,15 @@ g_gate_vlog(int priority, const char *message, va_list ap) default: prefix = "unknown"; } - - printf("%s: ", prefix); - vprintf(message, ap); - printf("\n"); + if (g_gate_logfd == -1) { + printf("%s: ", prefix); + vprintf(message, ap); + printf("\n"); + } else if (g_gate_verbose || priority != LOG_DEBUG) { + dprintf(g_gate_logfd, "%s: ", prefix); + vdprintf(g_gate_logfd, message, ap); + dprintf(g_gate_logfd, "\n"); + } } else { if (priority != LOG_DEBUG) vsyslog(priority, message, ap); diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index cc67ca49571f..422a60e0b7d5 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -97,6 +97,7 @@ struct g_gate_hdr { uint16_t gh_error; /* error value (0 if ok) */ } __packed; +void g_gate_open_log(const char *logfile); void g_gate_vlog(int priority, const char *message, va_list ap); void g_gate_log(int priority, const char *message, ...); void g_gate_xvlog(const char *message, va_list ap) __dead2; -- 2.32.0 From cba5aea71f500a065bfe6d9499775efad79cf224 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Thu, 30 Apr 2015 13:52:39 +0200 Subject: [PATCH 309/325] ggate[cd]: Add Jail and Capsicum support The capsicum support for ggatec is incomplete and only enabled if the -c flag is used as it currently prevents ggatec from reconnecting which is very inconvenient. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.8 | 8 +++ sbin/ggate/ggatec/ggatec.c | 37 ++++++++-- sbin/ggate/ggated/ggated.c | 5 ++ sbin/ggate/shared/ggate.c | 137 +++++++++++++++++++++++++++++++++++++ sbin/ggate/shared/ggate.h | 2 + 5 files changed, 182 insertions(+), 7 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.8 b/sbin/ggate/ggatec/ggatec.8 index ba3c9ecb6f0b..f4bf7cf43902 100644 --- a/sbin/ggate/ggatec/ggatec.8 +++ b/sbin/ggate/ggatec/ggatec.8 @@ -33,6 +33,7 @@ .Sh SYNOPSIS .Nm .Cm create +.Op Fl c .Op Fl n .Op Fl v .Op Fl o Cm ro | wo | rw @@ -48,6 +49,7 @@ .Ar path .Nm .Cm rescue +.Op Fl c .Op Fl n .Op Fl v .Op Fl o Cm ro | wo | rw @@ -104,6 +106,12 @@ providers. .Pp Available options: .Bl -tag -width ".Fl s Cm ro | wo | rw" +.It Fl c +Enter capsicum sandbox. +Currently this prevents +.Nm ggatec +from reconnecting which is somewhat inconvenient. +The flag will go away once this is fixed. .It Fl f Forcibly destroy .Nm ggate diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 6af026e80277..93da6bd1173b 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -53,6 +53,7 @@ #include #include #include +#include #include #include "ggate.h" @@ -62,6 +63,8 @@ static enum { UNSET, CREATE, DESTROY, LIST, RESCUE } action = UNSET; static const char *path = NULL; static const char *host = NULL; +static in_addr_t host_ip; +static const char *logfile = NULL; static int unit = G_GATE_UNIT_AUTO; static unsigned flags = 0; static int force = 0; @@ -76,6 +79,7 @@ static int sendfd, recvfd; static uint32_t token; static pthread_t sendtd, recvtd; static int reconnect; +static int drop_capabilities = 0; static void usage(void) @@ -369,7 +373,7 @@ handshake(int dir) */ bzero(&serv, sizeof(serv)); serv.sin_family = AF_INET; - serv.sin_addr.s_addr = g_gate_str2ip(host); + serv.sin_addr.s_addr = host_ip; if (serv.sin_addr.s_addr == INADDR_NONE) { g_gate_log(LOG_DEBUG, "Invalid IP/host name: %s.", host); return (-1); @@ -477,7 +481,7 @@ static void mydaemon(void) { - if (g_gate_verbose > 0) + if (logfile == NULL && g_gate_verbose > 0) return; if (daemon(0, 0) == 0) return; @@ -542,6 +546,10 @@ g_gatec_loop(void) signal(SIGUSR1, signop); for (;;) { g_gatec_start(); + + if (cap_sandboxed()) + g_gate_xlog("Got disconnected while being sandboxed."); + g_gate_log(LOG_NOTICE, "Disconnected [%s %s]. Connecting...", host, path); while (!g_gatec_connect()) { @@ -564,9 +572,6 @@ g_gatec_create(void) if (!g_gatec_connect()) g_gate_xlog("Cannot connect: %s.", strerror(errno)); - /* - * Ok, got both sockets, time to create provider. - */ memset(&ggioc, 0, sizeof(ggioc)); ggioc.gctl_version = G_GATE_VERSION; ggioc.gctl_mediasize = mediasize; @@ -589,6 +594,9 @@ g_gatec_create(void) } unit = ggioc.gctl_unit; + if (drop_capabilities) + g_gate_drop_capabilities(sendfd, recvfd); + mydaemon(); g_gatec_loop(); } @@ -601,6 +609,9 @@ g_gatec_rescue(void) if (!g_gatec_connect()) g_gate_xlog("Cannot connect: %s.", strerror(errno)); + if (drop_capabilities) + g_gate_drop_capabilities(sendfd, recvfd); + ggioc.gctl_version = G_GATE_VERSION; ggioc.gctl_unit = unit; ggioc.gctl_seq = 0; @@ -632,17 +643,21 @@ main(int argc, char *argv[]) int ch; char *p; - ch = getopt(argc, argv, "fl:no:p:q:R:S:s:t:T:u:v"); + ch = getopt(argc, argv, "cfl:no:p:q:R:S:s:t:T:u:v"); if (ch == -1) break; switch (ch) { + case 'c': + drop_capabilities = 1; + force = 1; + break; case 'f': if (action != DESTROY) usage(); force = 1; break; case 'l': - g_gate_open_log(optarg); + logfile = optarg; break; case 'n': if (action != CREATE && action != RESCUE) @@ -754,7 +769,11 @@ main(int argc, char *argv[]) g_gate_load_module(); g_gate_open_device(); host = argv[0]; + host_ip = g_gate_str2ip(host); path = argv[1]; + if (logfile != NULL) + g_gate_open_log(logfile); + g_gate_drop_privs("hast", host_ip); g_gatec_create(); break; case DESTROY: @@ -778,7 +797,11 @@ main(int argc, char *argv[]) } g_gate_open_device(); host = argv[0]; + host_ip = g_gate_str2ip(host); path = argv[1]; + if (logfile != NULL) + g_gate_open_log(logfile); + g_gate_drop_privs("hast", host_ip); g_gatec_rescue(); break; case UNSET: diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index f811b853f79c..46c3063b02b2 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -560,6 +560,11 @@ connection_launch(struct ggd_connection *conn) } g_gate_log(LOG_DEBUG, "Process created [%s].", conn->c_path); + if (getuid() == 0) + g_gate_drop_privs("hast", bindaddr); + + g_gate_drop_capabilities(conn->c_sendfd, conn->c_recvfd); + /* * Create condition variables and mutexes for in-queue and out-queue * synchronization. diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index 084db7304e53..4c0d3886f463 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -55,6 +55,11 @@ #include #include #include +#include +#include +#include +#include +#include #include #include "ggate.h" @@ -443,3 +448,135 @@ g_gate_cmd2str(int cmd) return ("unknown (invalid?) GGATE command"); } + +/* + * The functions below are based on drop_privs() from ../../hastd/subr.c + * + * Changes: + * - HAST_USER replaced with ggate_user option + * - pjdlog_* replaced with g_gate_xlog(). + * - Don't fall back to chroot if jailing fails. + */ +#define PJDLOG_VERIFY assert +void +g_gate_drop_privs(const char *ggate_user, in_addr_t jail_address) +{ + char jailhost[32]; + struct jail jailst; + struct passwd *pw; + uid_t ruid, euid, suid; + gid_t rgid, egid, sgid; + gid_t gidset[1]; + struct in_addr jail_ip; + /* + * According to getpwnam(3) we have to clear errno before calling the + * function to be able to distinguish between an error and missing + * entry (with is not treated as error by getpwnam(3)). + */ + errno = 0; + pw = getpwnam(ggate_user); + if (pw == NULL) { + if (errno != 0) { + g_gate_xlog("Unable to find info about '%s' user", + ggate_user); + } else { + g_gate_xlog("'%s' user doesn't exist.", ggate_user); + } + } + + jail_ip.s_addr = jail_address; + + bzero(&jailst, sizeof(jailst)); + jailst.version = JAIL_API_VERSION; + jailst.path = pw->pw_dir; + (void)snprintf(jailhost, sizeof(jailhost), "%s-jail", getprogname()); + jailst.hostname = jailhost; + jailst.jailname = NULL; + jailst.ip4s = 1; + jailst.ip4 = &jail_ip; + jailst.ip6s = 0; + jailst.ip6 = NULL; + if (jail(&jailst) == -1) { + g_gate_xlog("Unable to jail process in directory %s", pw->pw_dir); + } + PJDLOG_VERIFY(chdir("/") == 0); + gidset[0] = pw->pw_gid; + if (setgroups(1, gidset) == -1) { + g_gate_xlog("Unable to set groups to gid %u", + (unsigned int)pw->pw_gid); + } + if (setgid(pw->pw_gid) == -1) { + g_gate_xlog("Unable to set gid to %u", + (unsigned int)pw->pw_gid); + } + if (setuid(pw->pw_uid) == -1) { + g_gate_xlog("Unable to set uid to %u", + (unsigned int)pw->pw_uid); + } + + /* + * Better be sure that everything succeeded. + */ + PJDLOG_VERIFY(getresuid(&ruid, &euid, &suid) == 0); + PJDLOG_VERIFY(ruid == pw->pw_uid); + PJDLOG_VERIFY(euid == pw->pw_uid); + PJDLOG_VERIFY(suid == pw->pw_uid); + PJDLOG_VERIFY(getresgid(&rgid, &egid, &sgid) == 0); + PJDLOG_VERIFY(rgid == pw->pw_gid); + PJDLOG_VERIFY(egid == pw->pw_gid); + PJDLOG_VERIFY(sgid == pw->pw_gid); + PJDLOG_VERIFY(getgroups(0, NULL) == 1); + PJDLOG_VERIFY(getgroups(1, gidset) == 1); + PJDLOG_VERIFY(gidset[0] == pw->pw_gid); + + g_gate_log(LOG_DEBUG, "Privileges successfully dropped using " + "jail+setgid+setuid."); +} + +int +g_gate_drop_capabilities(int sendfd, int recvfd) +{ + cap_rights_t rights; + static const unsigned long ggatecmds[] = { + G_GATE_CMD_START, + G_GATE_CMD_DONE, + G_GATE_CMD_CANCEL, + }; + + if (cap_enter() != 0) { + g_gate_xlog("Failed to sandbox using capsicum"); + } + + cap_rights_init(&rights, CAP_PREAD, CAP_PWRITE); + if (cap_rights_limit(sendfd, &rights) == -1) { + g_gate_xlog("Unable to limit capability " + "rights on sendfd %d", sendfd); + } + if (cap_rights_limit(recvfd, &rights) == -1) { + g_gate_xlog("Unable to limit capability " + "rights on recvfd %d", recvfd); + } + + /* Only the client uses this. */ + if (g_gate_devfd != -1) { + cap_rights_init(&rights, CAP_IOCTL, CAP_PREAD, CAP_PWRITE); + if (cap_rights_limit(g_gate_devfd, &rights) == -1) { + g_gate_xlog("Unable to limit capability rights " + "to CAP_IOCTL on ggate descriptor"); + } + if (cap_ioctls_limit(g_gate_devfd, ggatecmds, + sizeof(ggatecmds) / sizeof(ggatecmds[0])) == -1) { + g_gate_xlog("Unable to limit allowed ggate ioctls"); + } + } + cap_rights_init(&rights, CAP_PWRITE); + if (g_gate_logfd != -1 && + cap_rights_limit(g_gate_logfd, &rights) == -1) { + g_gate_xlog("Unable to limit capability " + "rights on logfd %d", g_gate_logfd); + } + + g_gate_log(LOG_DEBUG, "Entered Capsicum sandbox"); + + return (0); +} diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 422a60e0b7d5..37049e431a83 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -117,6 +117,8 @@ void g_gate_list(int unit, int verbose); #endif in_addr_t g_gate_str2ip(const char *str); const char *g_gate_cmd2str(int cmd); +void g_gate_drop_privs(const char *ggate_user, in_addr_t jail_address); +int g_gate_drop_capabilities(int sendfd, int recvfd); /* * g_gate_swap2h_* - functions swap bytes to host byte order (from big endian). -- 2.32.0 From 979b2384e31e38e505b7913981ceeb86c5e0b42b Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 28 Apr 2015 13:02:25 +0200 Subject: [PATCH 310/325] Bump GGATE_VERSION due to FLUSH and DELETE support and various bug fixes Unpatched ggate[cd] versions may cause data corruption so we no longer want to speak to them. Obtained from: ElectroBSD --- sbin/ggate/shared/ggate.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 37049e431a83..5d3a9bd6025b 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -42,7 +42,7 @@ #define G_GATE_TIMEOUT 0 #define GGATE_MAGIC "GEOM_GATE " -#define GGATE_VERSION 0 +#define GGATE_VERSION 1 #define GGATE_FLAG_RDONLY 0x0001 #define GGATE_FLAG_WRONLY 0x0002 -- 2.32.0 From c45ec31f3109338c48d7454b0a37b84746aed964 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 9 Aug 2015 15:20:48 +0200 Subject: [PATCH 311/325] ggate: Use dedicated users for ggatec and ggated Obtained from: ElectroBSD --- etc/group | 2 ++ etc/master.passwd | 2 ++ sbin/ggate/ggatec/ggatec.c | 4 ++-- sbin/ggate/ggated/ggated.c | 2 +- 4 files changed, 7 insertions(+), 3 deletions(-) diff --git a/etc/group b/etc/group index da928bf12c77..82cf3b6384c5 100644 --- a/etc/group +++ b/etc/group @@ -33,6 +33,8 @@ ntpd:*:123: _ypldap:*:160: _tor:*:256: hast:*:845: +ggatec:*:846: +ggated:*:847: tests:*:977: nogroup:*:65533: nobody:*:65534: diff --git a/etc/master.passwd b/etc/master.passwd index 597aebe125dc..d1ad7c159d50 100644 --- a/etc/master.passwd +++ b/etc/master.passwd @@ -26,5 +26,7 @@ ntpd:*:123:123::0:0:NTP Daemon:/var/db/ntp:/usr/sbin/nologin _ypldap:*:160:160::0:0:YP LDAP unprivileged user:/var/empty:/usr/sbin/nologin _tor:*:256:256::0:0:Onion delivery agent:/nonexistent:/usr/sbin/nologin hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin +ggatec:*:846:846::0:0:ggatec unprivileged user:/var/empty:/usr/sbin/nologin +ggated:*:847:847::0:0:ggated unprivileged user:/var/empty:/usr/sbin/nologin tests:*:977:977::0:0:Unprivileged user for tests:/nonexistent:/usr/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index 93da6bd1173b..ee719aabf17c 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -773,7 +773,7 @@ main(int argc, char *argv[]) path = argv[1]; if (logfile != NULL) g_gate_open_log(logfile); - g_gate_drop_privs("hast", host_ip); + g_gate_drop_privs("ggatec", host_ip); g_gatec_create(); break; case DESTROY: @@ -801,7 +801,7 @@ main(int argc, char *argv[]) path = argv[1]; if (logfile != NULL) g_gate_open_log(logfile); - g_gate_drop_privs("hast", host_ip); + g_gate_drop_privs("ggatec", host_ip); g_gatec_rescue(); break; case UNSET: diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 46c3063b02b2..82df8a8aa391 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -561,7 +561,7 @@ connection_launch(struct ggd_connection *conn) g_gate_log(LOG_DEBUG, "Process created [%s].", conn->c_path); if (getuid() == 0) - g_gate_drop_privs("hast", bindaddr); + g_gate_drop_privs("ggated", bindaddr); g_gate_drop_capabilities(conn->c_sendfd, conn->c_recvfd); -- 2.32.0 From a69bdf412268f8baf777d8eeac3ffbff9352e57e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Fri, 28 Apr 2017 22:46:35 +0200 Subject: [PATCH 312/325] ggate: Increase maximum BIO length to 1MB This is required for ggatec/ggated to work with ZFS pools with recordsize=1m. XXX: Incrementing the zeros buffer shouldn't be necessary and is unlikely to speed up the trimming. Obtained from: ElectroBSD --- sbin/ggate/ggatec/ggatec.c | 6 +++--- sbin/ggate/ggated/ggated.c | 17 ++++++++++------- sbin/ggate/shared/ggate.h | 2 ++ 3 files changed, 15 insertions(+), 10 deletions(-) diff --git a/sbin/ggate/ggatec/ggatec.c b/sbin/ggate/ggatec/ggatec.c index ee719aabf17c..37f03f4423ec 100644 --- a/sbin/ggate/ggatec/ggatec.c +++ b/sbin/ggate/ggatec/ggatec.c @@ -102,7 +102,7 @@ send_thread(void *arg __unused) { struct g_gate_ctl_io ggio; struct g_gate_hdr hdr; - char buf[MAXPHYS]; + char buf[GGATE_MAX_BIO_LENGTH]; ssize_t data; int error; @@ -170,7 +170,7 @@ send_thread(void *arg __unused) } /* Don't send requests for more data than we can handle the response for! */ - if (ggio.gctl_length > MAXPHYS) { + if (ggio.gctl_length > GGATE_MAX_BIO_LENGTH) { g_gate_log(LOG_ERR, "Request too big: %zd", ggio.gctl_length); ggio.gctl_error = EOPNOTSUPP; g_gate_ioctl(G_GATE_CMD_DONE, &ggio); @@ -222,7 +222,7 @@ recv_thread(void *arg __unused) { struct g_gate_ctl_io ggio; struct g_gate_hdr hdr; - char buf[MAXPHYS]; + char buf[GGATE_MAX_BIO_LENGTH]; ssize_t data; g_gate_log(LOG_NOTICE, "%s: started!", __func__); diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 82df8a8aa391..132998bab152 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -707,12 +707,14 @@ recv_thread(void *arg) /* * Limit the amount of memory we allocate on behalf of - * the client. MAXPHYS is the hard limit in ggatec, - * values above it are thus pretty suspicious. + * the client. GGATE_MAX_BIO_LENGTH is the hard limit in + * ggatec, values above it are thus pretty suspicious. */ - if (req->r_length > MAXPHYS && req->r_cmd != GGATE_CMD_DELETE) { - g_gate_xlog("Request length above MAXPHYS: %u > %u", - (unsigned)req->r_length, MAXPHYS); + if (req->r_length > GGATE_MAX_BIO_LENGTH && + req->r_cmd != GGATE_CMD_DELETE) { + g_gate_xlog("Request length above " + "GGATE_MAX_BIO_LENGTH: %u > %u", + (unsigned)req->r_length, GGATE_MAX_BIO_LENGTH); } /* @@ -749,7 +751,7 @@ recv_thread(void *arg) static ssize_t delete_range(int fd, size_t length, off_t offset) { - static char zeros[MAXPHYS]; + static char zeros[GGATE_MAX_BIO_LENGTH]; size_t written; written = 0; @@ -761,7 +763,8 @@ delete_range(int fd, size_t length, off_t offset) size_t chunk_size; bytes_left = length - written; - chunk_size = bytes_left > MAXPHYS ? MAXPHYS : bytes_left; + chunk_size = bytes_left > sizeof(zeros) ? + sizeof(zeros) : bytes_left; ret = pwrite(fd, zeros, chunk_size, offset + written); if (ret == -1) return (written); diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index 5d3a9bd6025b..bb6ddc4ac8f8 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -60,6 +60,8 @@ #define GGATE_CMD_FLUSH 2 #define GGATE_CMD_DELETE 3 +#define GGATE_MAX_BIO_LENGTH 1048576 + extern int g_gate_devfd; extern int g_gate_verbose; -- 2.32.0 From 82ee8d12b0286ba451ed380bd03adbc9ff99e05d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 13 Nov 2021 09:56:26 +0100 Subject: [PATCH 313/325] ggated: Close the pid file and directory after forking ... so we're allowed to go to jail without having to set kern.pwd_chroot_chdir_check_open_directories=0 again. See also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259770 Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 132998bab152..74114a292f6b 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -95,6 +95,7 @@ struct ggd_export { SLIST_ENTRY(ggd_export) e_next; }; +static struct pidfh *pfh; static const char *exports_file = GGATED_EXPORT_FILE; static int got_sighup = 0; static in_addr_t bindaddr; @@ -560,6 +561,9 @@ connection_launch(struct ggd_connection *conn) } g_gate_log(LOG_DEBUG, "Process created [%s].", conn->c_path); + if (pidfile_close(pfh) == -1) + g_gate_xlog("pidfile_close(): %s.", strerror(errno)); + if (getuid() == 0) g_gate_drop_privs("ggated", bindaddr); @@ -1066,7 +1070,6 @@ int main(int argc, char *argv[]) { const char *ggated_pidfile = _PATH_VARRUN "/ggated.pid"; - struct pidfh *pfh; struct sockaddr_in serv; struct sockaddr from; socklen_t fromlen; -- 2.32.0 From 7dce4298266ea75081d3c4a9af1466fbe248c37d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sat, 13 Nov 2021 10:45:08 +0100 Subject: [PATCH 314/325] ggated: Let connection_launch() close the accepting socket after forking There's no reason why the children should have access to it. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 74114a292f6b..2627b757d635 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -547,7 +547,7 @@ connection_ready(struct ggd_connection *conn) } static void -connection_launch(struct ggd_connection *conn) +connection_launch(struct ggd_connection *conn, int sfd) { pthread_t td; int error, pid; @@ -563,6 +563,8 @@ connection_launch(struct ggd_connection *conn) if (pidfile_close(pfh) == -1) g_gate_xlog("pidfile_close(): %s.", strerror(errno)); + if (close(sfd) == -1) + g_gate_xlog("close(sfd): %s.", strerror(errno)); if (getuid() == 0) g_gate_drop_privs("ggated", bindaddr); @@ -950,7 +952,7 @@ log_connection(struct sockaddr *from) } static int -handshake(struct sockaddr *from, int sfd) +handshake(struct sockaddr *from, int sfd, int listen_fd) { struct g_gate_version ver; struct g_gate_cinit cinit; @@ -1053,7 +1055,7 @@ handshake(struct sockaddr *from, int sfd) } if (connection_ready(conn)) { - connection_launch(conn); + connection_launch(conn, listen_fd); connection_remove(conn); } return (1); @@ -1179,7 +1181,7 @@ main(int argc, char *argv[]) exports_get(); } - if (!handshake(&from, tmpsfd)) + if (!handshake(&from, tmpsfd, sfd)) close(tmpsfd); } close(sfd); -- 2.32.0 From 40c557c9c8c8fe3b7bb00db663de6581f9595ae9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Nov 2021 15:06:57 +0100 Subject: [PATCH 315/325] ggated: Add undocumented -j option to test jailing Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 2627b757d635..9a4e7838d0e4 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1078,10 +1078,11 @@ main(int argc, char *argv[]) pid_t otherpid; int ch, sfd, tmpsfd; unsigned port; + int g_gate_jail_test = 0; bindaddr = htonl(INADDR_ANY); port = G_GATE_PORT; - while ((ch = getopt(argc, argv, "a:hnp:F:R:S:v")) != -1) { + while ((ch = getopt(argc, argv, "a:hnjp:F:R:S:v")) != -1) { switch (ch) { case 'a': bindaddr = g_gate_str2ip(optarg); @@ -1093,6 +1094,9 @@ main(int argc, char *argv[]) case 'F': ggated_pidfile = optarg; break; + case 'j': + g_gate_jail_test = 1; + break; case 'n': nagle = 0; break; @@ -1127,7 +1131,8 @@ main(int argc, char *argv[]) if (argv[0] != NULL) exports_file = argv[0]; - exports_get(); + if (!g_gate_jail_test) + exports_get(); pfh = pidfile_open(ggated_pidfile, 0600, &otherpid); if (pfh == NULL) { @@ -1168,6 +1173,13 @@ main(int argc, char *argv[]) signal(SIGHUP, huphandler); + if (g_gate_jail_test) { + pidfile_close(pfh); + g_gate_drop_privs("ggated", bindaddr); + pidfile_remove(pfh); + exit(EXIT_SUCCESS); + } + for (;;) { fromlen = sizeof(from); tmpsfd = accept(sfd, &from, &fromlen); -- 2.32.0 From 1c3583b49c311a8f21552f1fef6b20cccd70089c Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 1 Nov 2021 16:32:40 +0100 Subject: [PATCH 316/325] ggated: Add undocumented -J flag to disable jailing Should work around: fk@r500 ~ $sudo ggated -v -j info: Reading exports file (/etc/gg.exports). debug: Added 127.0.0.1/32 /dev/zvol/r500/ggated/t520.eli RW to exports list. info: Exporting 1 object(s). info: Listen on port: 3080. error: Unable to jail process in directory /var/empty error: Exiting. Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 5 ++++- sbin/ggate/shared/ggate.c | 3 ++- sbin/ggate/shared/ggate.h | 1 + 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 9a4e7838d0e4..779e5c224dbe 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1082,7 +1082,7 @@ main(int argc, char *argv[]) bindaddr = htonl(INADDR_ANY); port = G_GATE_PORT; - while ((ch = getopt(argc, argv, "a:hnjp:F:R:S:v")) != -1) { + while ((ch = getopt(argc, argv, "a:hnjJp:F:R:S:v")) != -1) { switch (ch) { case 'a': bindaddr = g_gate_str2ip(optarg); @@ -1094,6 +1094,9 @@ main(int argc, char *argv[]) case 'F': ggated_pidfile = optarg; break; + case 'J': + g_gate_no_jailing = 1; + break; case 'j': g_gate_jail_test = 1; break; diff --git a/sbin/ggate/shared/ggate.c b/sbin/ggate/shared/ggate.c index 4c0d3886f463..05a2cdcd1f83 100644 --- a/sbin/ggate/shared/ggate.c +++ b/sbin/ggate/shared/ggate.c @@ -66,6 +66,7 @@ int g_gate_devfd = -1; +int g_gate_no_jailing = 0; int g_gate_verbose = 0; static int g_gate_logfd = -1; @@ -496,7 +497,7 @@ g_gate_drop_privs(const char *ggate_user, in_addr_t jail_address) jailst.ip4 = &jail_ip; jailst.ip6s = 0; jailst.ip6 = NULL; - if (jail(&jailst) == -1) { + if (!g_gate_no_jailing && jail(&jailst) == -1) { g_gate_xlog("Unable to jail process in directory %s", pw->pw_dir); } PJDLOG_VERIFY(chdir("/") == 0); diff --git a/sbin/ggate/shared/ggate.h b/sbin/ggate/shared/ggate.h index bb6ddc4ac8f8..0a648d5e3513 100644 --- a/sbin/ggate/shared/ggate.h +++ b/sbin/ggate/shared/ggate.h @@ -63,6 +63,7 @@ #define GGATE_MAX_BIO_LENGTH 1048576 extern int g_gate_devfd; +extern int g_gate_no_jailing; extern int g_gate_verbose; extern int nagle; -- 2.32.0 From 2bddd75f644aa02da9b78389a1891ef46e172208 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Sun, 19 Apr 2015 22:58:49 +0200 Subject: [PATCH 317/325] ggated: Default to listening to 127.0.0.1 only Obtained from: ElectroBSD --- sbin/ggate/ggated/ggated.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/ggate/ggated/ggated.c b/sbin/ggate/ggated/ggated.c index 779e5c224dbe..5c55a246207c 100644 --- a/sbin/ggate/ggated/ggated.c +++ b/sbin/ggate/ggated/ggated.c @@ -1080,7 +1080,7 @@ main(int argc, char *argv[]) unsigned port; int g_gate_jail_test = 0; - bindaddr = htonl(INADDR_ANY); + bindaddr = g_gate_str2ip("127.0.0.1"); port = G_GATE_PORT; while ((ch = getopt(argc, argv, "a:hnjJp:F:R:S:v")) != -1) { switch (ch) { -- 2.32.0 From 1fd14d7cde18cda72abb95cb3bc25144a77b7ae6 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Mon, 29 Nov 2021 08:50:52 +0100 Subject: [PATCH 318/325] release/scripts/strip-freebsd.sh: Ditch *.ucode files *.ucode files were added for iwlwifi in f4c129f5fb1669b28 but haven't been MFC'd to stable/12 (yet). Obtained from: ElectroBSD --- release/scripts/strip-freebsd.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/release/scripts/strip-freebsd.sh b/release/scripts/strip-freebsd.sh index 556b2ba4e365..fb0438300078 100755 --- a/release/scripts/strip-freebsd.sh +++ b/release/scripts/strip-freebsd.sh @@ -83,6 +83,9 @@ get_usb_firmware_files() { # by non-free dependencies get_unfree_files() { find sys/ -name "*.uu" + # *.ucode files were added for iwlwifi in f4c129f5fb1669b28 + # which hasn't been MFC'd to stable/12 (yet). + find sys/ -name "*.ucode" get_dev_ice_files get_dev_imx_files get_dev_qatfw_files -- 2.32.0 From 17b404bc507ab0f0ce17c5b25ba25e9f4e97ec7e Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 12:50:24 +0100 Subject: [PATCH 319/325] stand/lua/gfx-beastie.lua: Replace fork with toilet brush Obtained from: ElectroBSD --- stand/lua/gfx-beastie.lua | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/stand/lua/gfx-beastie.lua b/stand/lua/gfx-beastie.lua index f88f8c8dd306..dc69c0413120 100644 --- a/stand/lua/gfx-beastie.lua +++ b/stand/lua/gfx-beastie.lua @@ -41,9 +41,9 @@ return { " (_.) _ ) /", " `.___/` /", " `-----' /", - " \027[33m<----.\027[31m __ / __ \\", - " \027[33m<----|====\027[31mO)))\027[33m==\027[31m) \\) /\027[33m====|", - " \027[33m<----'\027[31m `--' `.__,' \\", + " \027[33m######\027[31m __ / __ \\", + " \027[33m######====\027[31mO)))\027[33m==\027[31m) \\) /\027[33m====|", + " \027[33m######\027[31m `--' `.__,' \\", " | |", " \\ / /\\", " \027[36m______\027[31m( (_ / \\______/", -- 2.32.0 From e6c3d1b56763c9a4ba5bb888627934ffc33602e9 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 12:51:25 +0100 Subject: [PATCH 320/325] stand/lua/gfx-beastiebw.lua: Replace fork with toilet brush Obtained from: ElectroBSD --- stand/lua/gfx-beastiebw.lua | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/stand/lua/gfx-beastiebw.lua b/stand/lua/gfx-beastiebw.lua index 5895b2436542..8ae8599c3b57 100644 --- a/stand/lua/gfx-beastiebw.lua +++ b/stand/lua/gfx-beastiebw.lua @@ -41,9 +41,9 @@ return { " (_.) _ ) /", " `.___/` /", " `-----' /", - " <----. __ / __ \\", - " <----|====O)))==) \\) /====|", - " <----' `--' `.__,' \\", + " ###### __ / __ \\", + " ######====O)))==) \\) /====|", + " ###### `--' `.__,' \\", " | |", " \\ / /\\", " ______( (_ / \\______/", -- 2.32.0 From a5c02a4f8ed3de5cde536ceb6162a750d79c1f5a Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 13:03:37 +0100 Subject: [PATCH 321/325] stand/defaults/loader.conf: Set beastie_disable="NO" Obtained from: ElectroBSD --- stand/defaults/loader.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/defaults/loader.conf b/stand/defaults/loader.conf index a1eb189db7ac..658c6a9bafff 100644 --- a/stand/defaults/loader.conf +++ b/stand/defaults/loader.conf @@ -93,7 +93,7 @@ audit_event_type="etc_security_audit_event" #bootlock_password="" # Prevent booting (see check-password.4th(8)) #geom_eli_passphrase_prompt="NO" # Prompt for geli(8) passphrase to mount root bootenv_autolist="YES" # Auto populate the list of ZFS Boot Environments -#beastie_disable="NO" # Turn the beastie boot menu on and off +beastie_disable="NO" # Turn the beastie boot menu on and off efi_max_resolution="1x1" # Set the max resolution for EFI loader to use: # 480p, 720p, 1080p, 2160p/4k, 5k, or specify # WidthxHeight (e.g. 1920x1080) -- 2.32.0 From a0aa0cdb5570008dc053dc3b0d17735a977dbc76 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 13:50:06 +0100 Subject: [PATCH 322/325] stand/lua/drawer.lua: Welcome users to ElectroBSD instead of FreeBSD Obtained from: ElectroBSD --- stand/lua/drawer.lua | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stand/lua/drawer.lua b/stand/lua/drawer.lua index 499b59e37daf..59a72f26bcd7 100644 --- a/stand/lua/drawer.lua +++ b/stand/lua/drawer.lua @@ -250,7 +250,7 @@ local function drawbox() end local menu_header = loader.getenv("loader_menu_title") or - "Welcome to FreeBSD" + "Welcome to ElectroBSD" local menu_header_align = loader.getenv("loader_menu_title_align") local menu_header_x -- 2.32.0 From 9943858d63c3e2efb658ac7b2328ad9ad9bea78d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 12:57:20 +0100 Subject: [PATCH 323/325] stand/lua/drawer.lua: Change the OS name in the banner to ElectroBSD While at it, suggest to resist unlawful police activities (German). Obtained from: ElectroBSD --- stand/lua/drawer.lua | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/stand/lua/drawer.lua b/stand/lua/drawer.lua index 59a72f26bcd7..f1a8ea7a47f9 100644 --- a/stand/lua/drawer.lua +++ b/stand/lua/drawer.lua @@ -341,14 +341,17 @@ local function drawlogo() draw(x, y, logodef.graphic) end +-- XXX: The variable should probably be renamed to 'electrobsd_brand' fbsd_brand = { -" ______ ____ _____ _____ ", -" | ____| | _ \\ / ____| __ \\ ", -" | |___ _ __ ___ ___ | |_) | (___ | | | |", -" | ___| '__/ _ \\/ _ \\| _ < \\___ \\| | | |", -" | | | | | __/ __/| |_) |____) | |__| |", -" | | | | | | || | | |", -" |_| |_| \\___|\\___||____/|_____/|_____/ " +" ______ _ _ ____ _____ _____", +" | ____| | | | | _ \\ / ____| __ \\", +" | |__ | | ___ ___| |_ _ __ ___ | |_) | (___ | | | |", +" | __| | |/ _ \\/ __| __| '__/ _ \\| _ < \\___ \\| | | |", +" | |____| | __/ (__| |_| | | (_) | |_) |____) | |__| |", +" |______|_|\\___|\\___|\\__|_| \\___/|____/|_____/|_____/", +" Polizei-Willkuer in Deinem Land? Das erfordert Widerstand!", +" Zu Risiken und Nebenwirkungen fragen Sie Ihren Anwalt oder", +" die Rote Hilfe." } none = {""} -- 2.32.0 From 817fe7d130ce3f4345981b6e28a62f5177cad95d Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Tue, 30 Nov 2021 13:46:49 +0100 Subject: [PATCH 324/325] stand/lua/drawer.lua: Think of the children and default to the beastie logo ... instead of the 'sex toy'. Obtained from: ElectroBSD --- stand/lua/drawer.lua | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/stand/lua/drawer.lua b/stand/lua/drawer.lua index f1a8ea7a47f9..35de0fb3827e 100644 --- a/stand/lua/drawer.lua +++ b/stand/lua/drawer.lua @@ -420,8 +420,8 @@ shift = default_shift -- Module exports drawer.default_brand = 'fbsd' -drawer.default_color_logodef = 'orb' -drawer.default_bw_logodef = 'orbbw' +drawer.default_color_logodef = 'beastie' +drawer.default_bw_logodef = 'beastiebw' -- For when things go terribly wrong; this def should be present here in the -- drawer module in case it's a filesystem issue. drawer.default_fallback_logodef = 'none' -- 2.32.0 From 832b8f2448ba1112ce8b840030dd8b5d14c56714 Mon Sep 17 00:00:00 2001 From: Fabian Keil Date: Wed, 1 Dec 2021 18:15:36 +0100 Subject: [PATCH 325/325] Add reproduce.conf for ElectroBSD-20211130-817fe7d130ce Expected checksums when reproducing on amd64: SHA256 (/usr/obj/usr/src/ElectroBSD-20211130-817fe7d130ce-j1/src.txz) = 317b0a96ab6e67a2cb09d0dd6c393fb9c4de4a7ddf1ba7ecc098a6a89a81baaa SHA256 (/usr/obj/usr/src/ElectroBSD-20211130-817fe7d130ce-j1/MANIFEST) = 92d5e114a65bc3f1ccced96fc5cbc200388979c3afdde1d7d03afbe649d9d087 SHA256 (/usr/obj/usr/src/ElectroBSD-20211130-817fe7d130ce-j1/kernel.txz) = 92fa5bceb56ce5ff05cb522555f74202d77e91cf2426f49d1599086b95bd1d41 SHA256 (/usr/obj/usr/src/ElectroBSD-20211130-817fe7d130ce-j1/kernel-dbg.txz) = 999c39fd5d3734facbbb543eb36d224972232f4e4c01633552cc40679e6ec7c3 SHA256 (/usr/obj/usr/src/ElectroBSD-20211130-817fe7d130ce-j1/base-dbg.txz) = a5ba30e127cfa6f3bcc66e42a8271820b3b5335465d343266af5e31ac1df67c8 SHA256 (/usr/obj/usr/src/ElectroBSD-20211130-817fe7d130ce-j1/tests.txz) = f7c320b4689493b5886d173056664e1a0726054e175e31a2cf01ade9439dbda1 SHA256 (/usr/obj/usr/src/ElectroBSD-20211130-817fe7d130ce-j1/base.txz) = fce167bc4a675a735843910b7c6e7fc243dcf90c49fb072c3be147932b6b053a The expected verbose output of image-checksum.sh is: gpart checksum: 63089c3730bb9a55342f1a71132bc2fad29ab6b0b5746e3e13ea0ee2208fd9e4 Boot code checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 mtree checksum: 67ca25a06160e79acb8160f74c49b33c40b7a31f2abcfe4c801a7cf235594007 ls checksum: f9b944005ac17ecea86f354a19e7f41e7037bb72d0cbae1eb46148c2f98c4350 Partial image checksum for ElectroBSD-20211130-817fe7d130ce.img: a9e1d8158b47f0a5c5636cb3aadcad916c13e002f04d7c3d634a7884f501d169 Obtained from: ElectroBSD --- reproduce.conf | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 reproduce.conf diff --git a/reproduce.conf b/reproduce.conf new file mode 100644 index 000000000000..d761c29061c2 --- /dev/null +++ b/reproduce.conf @@ -0,0 +1,2 @@ +BUILD=ElectroBSD-817fe7d130ce +EPOCH=1608157520 -- 2.32.0