remaining steps for time64 switchover

Submitted by Rich Felker on Oct. 21, 2019, 2:46 a.m.

Details

Message ID 20191021024643.GA6192@brightrain.aerifal.cx
State New
Series "remaining steps for time64 switchover"
Headers show

Commit Message

Rich Felker Oct. 21, 2019, 2:46 a.m.
The attached patch series on top of present git master (commit
9b2921bea1d5017832e1b45d1fd64220047a9802) should contain all changes
needed for fully working time64 on 32-bit archs, in a form that's
plausibly ready for commit (no makeshift hacks just to get things
demonstrably working). The one omission I'm aware of is what to do
with struct utmpx, which is not actually used at present in any libc
interfaces and thus not part of the ABI surface of libc. That will be
addressed in a separate thread.

Comments and basic testing are welcome at this point. It should be
possible to build for any of the 32-bit archs, but I have only tested
build for a few and only tested execution on i386 and sh.

Some useful checks for anyone wanting to help test, especially on the
more obscure archs:

- Any new failures in libc-test?

- Any failures when subbing the new libc.so/ldso into an existing
  ecosystem of existing (time32) software?

- Anything look odd about time-related types? timeval/timespec members
  at positions that aren't naturally aligned?

- Does software built against new libc headers basically work?

- Does it work with clock set post-2038?

Being that only the final switchover patches are a big step to take,
I'll probably start pushing the earlier patches in this series pretty
soon, but want to give a little time for more eyes on them.

Note that the final switchovers are split out by arch right now,
mainly because that was the way that made sense to create them (sed to
replace arch name, apply to next arch, fix rejects manually) but also
because it helps compare/review them against each other. It would be
possible to squash them together for final merge but I probably won't.

Rich
From 31283f25bcfa71d3eeaabc591d0eb3e2960980e2 Mon Sep 17 00:00:00 2001
From: Rich Felker <dalias@aerifal.cx>
Date: Wed, 31 Jul 2019 15:24:58 -0400
Subject: [PATCH 01/17] add time64 symbol name redirects to public headers,
 under arch control

a _REDIR_TIME64 macro is introduced, which the arch's alltypes.h is
expected to define, to control redirection of symbol names for
interfaces that involve time_t and derived types. this ensures that
object files will only be linked to libc interfaces matching the ABI
whose headers they were compiled against.

along with time32 compat shims, which will be introduced separately,
the redirection also makes it possible for a single libc (static or
shared) to be used with object files produced with either the old
(32-bit time_t) headers or the new ones after 64-bit time_t switchover
takes place. mixing of such object files (or shared libraries) in the
same program will also be possible, but must be done with care; ABI
between libc and a consumer of the libc interfaces is guaranteed to
match by the the symbol name redirection, but pairwise ABI between
consumers of libc that define interfaces between each other in terms
of time_t is not guaranteed to match.

this change adds a dependency on an additional "GNU C" feature to the
public headers for existing 32-bit archs, which is generally
undesirable; however, the feature is one which glibc has depended on
for a long time, and thus which any viable alternative compiler is
going to need to provide. 64-bit archs are not affected, nor will
future 32-bit archs be, regardless of whether they are "new" on the
kernel side (e.g. riscv32) or just newly-added (e.g. a new sparc or
xtensa port). the same applies to newly-added ABIs for existing
machine-level archs.
---
 include/aio.h          |  4 ++++
 include/features.h     |  2 ++
 include/mqueue.h       |  5 +++++
 include/poll.h         |  6 ++++++
 include/pthread.h      | 10 ++++++++++
 include/sched.h        |  4 ++++
 include/semaphore.h    |  4 ++++
 include/signal.h       |  8 ++++++++
 include/sys/resource.h |  4 ++++
 include/sys/select.h   |  5 +++++
 include/sys/sem.h      |  6 ++++++
 include/sys/socket.h   |  6 ++++++
 include/sys/stat.h     |  9 +++++++++
 include/sys/time.h     | 14 ++++++++++++++
 include/sys/timeb.h    |  6 ++++++
 include/sys/timerfd.h  |  5 +++++
 include/sys/timex.h    |  5 +++++
 include/sys/wait.h     |  7 +++++++
 include/threads.h      |  6 ++++++
 include/time.h         | 28 ++++++++++++++++++++++++++++
 include/utime.h        |  6 ++++++
 21 files changed, 150 insertions(+)

Patch hide | download patch | download mbox

diff --git a/include/aio.h b/include/aio.h
index 19bc28a9..453c41b7 100644
--- a/include/aio.h
+++ b/include/aio.h
@@ -62,6 +62,10 @@  int lio_listio(int, struct aiocb *__restrict const *__restrict, int, struct sige
 #define off64_t off_t
 #endif
 
+#if _REDIR_TIME64
+__REDIR(aio_suspend, __aio_suspend_time64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/features.h b/include/features.h
index f4d651ef..85cfb72a 100644
--- a/include/features.h
+++ b/include/features.h
@@ -35,4 +35,6 @@ 
 #define _Noreturn
 #endif
 
+#define __REDIR(x,y) __typeof__(x) x __asm__(#y)
+
 #endif
diff --git a/include/mqueue.h b/include/mqueue.h
index f5cbe796..0c807ea0 100644
--- a/include/mqueue.h
+++ b/include/mqueue.h
@@ -30,6 +30,11 @@  ssize_t mq_timedreceive(mqd_t, char *__restrict, size_t, unsigned *__restrict, c
 int mq_timedsend(mqd_t, const char *, size_t, unsigned, const struct timespec *);
 int mq_unlink(const char *);
 
+#if _REDIR_TIME64
+__REDIR(mq_timedreceive, __mq_timedreceive_time64);
+__REDIR(mq_timedsend, __mq_timedsend_time64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/poll.h b/include/poll.h
index daccc760..472e4b84 100644
--- a/include/poll.h
+++ b/include/poll.h
@@ -44,6 +44,12 @@  int poll (struct pollfd *, nfds_t, int);
 int ppoll(struct pollfd *, nfds_t, const struct timespec *, const sigset_t *);
 #endif
 
+#if _REDIR_TIME64
+#ifdef _GNU_SOURCE
+__REDIR(ppoll, __ppoll_time64);
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/pthread.h b/include/pthread.h
index e238321b..984db680 100644
--- a/include/pthread.h
+++ b/include/pthread.h
@@ -224,6 +224,16 @@  int pthread_tryjoin_np(pthread_t, void **);
 int pthread_timedjoin_np(pthread_t, void **, const struct timespec *);
 #endif
 
+#if _REDIR_TIME64
+__REDIR(pthread_mutex_timedlock, __pthread_mutex_timedlock_time64);
+__REDIR(pthread_cond_timedwait, __pthread_cond_timedwait_time64);
+__REDIR(pthread_rwlock_timedrdlock, __pthread_rwlock_timedrdlock_time64);
+__REDIR(pthread_rwlock_timedwrlock, __pthread_rwlock_timedwrlock_time64);
+#ifdef _GNU_SOURCE
+__REDIR(pthread_timedjoin_np, __pthread_timedjoin_np_time64);
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sched.h b/include/sched.h
index 7e470d3a..c3a8d49a 100644
--- a/include/sched.h
+++ b/include/sched.h
@@ -133,6 +133,10 @@  __CPU_op_func_S(XOR, ^)
 
 #endif
 
+#if _REDIR_TIME64
+__REDIR(sched_rr_get_interval, __sched_rr_get_interval_time64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/semaphore.h b/include/semaphore.h
index 277c47d6..3690f496 100644
--- a/include/semaphore.h
+++ b/include/semaphore.h
@@ -29,6 +29,10 @@  int    sem_trywait(sem_t *);
 int    sem_unlink(const char *);
 int    sem_wait(sem_t *);
 
+#if _REDIR_TIME64
+__REDIR(sem_timedwait, __sem_timedwait_time64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/signal.h b/include/signal.h
index 5c48cb83..fbdf667b 100644
--- a/include/signal.h
+++ b/include/signal.h
@@ -271,6 +271,14 @@  typedef int sig_atomic_t;
 void (*signal(int, void (*)(int)))(int);
 int raise(int);
 
+#if _REDIR_TIME64
+#if defined(_POSIX_SOURCE) || defined(_POSIX_C_SOURCE) \
+ || defined(_XOPEN_SOURCE) || defined(_GNU_SOURCE) \
+ || defined(_BSD_SOURCE)
+__REDIR(sigtimedwait, __sigtimedwait_time64);
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/resource.h b/include/sys/resource.h
index 70d793d5..e0c86ae3 100644
--- a/include/sys/resource.h
+++ b/include/sys/resource.h
@@ -104,6 +104,10 @@  int prlimit(pid_t, int, const struct rlimit *, struct rlimit *);
 #define rlim64_t rlim_t
 #endif
 
+#if _REDIR_TIME64
+__REDIR(getrusage, __getrusage_time64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/select.h b/include/sys/select.h
index d34cbf10..b3bab1d5 100644
--- a/include/sys/select.h
+++ b/include/sys/select.h
@@ -35,6 +35,11 @@  int pselect (int, fd_set *__restrict, fd_set *__restrict, fd_set *__restrict, co
 #define NFDBITS (8*(int)sizeof(long))
 #endif
 
+#if _REDIR_TIME64
+__REDIR(select, __select_time64);
+__REDIR(pselect, __pselect_time64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/sem.h b/include/sys/sem.h
index e6161e51..a747784e 100644
--- a/include/sys/sem.h
+++ b/include/sys/sem.h
@@ -60,6 +60,12 @@  int semop(int, struct sembuf *, size_t);
 int semtimedop(int, struct sembuf *, size_t, const struct timespec *);
 #endif
 
+#if _REDIR_TIME64
+#ifdef _GNU_SOURCE
+__REDIR(semtimedop, __semtimedop_time64);
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/socket.h b/include/sys/socket.h
index 8692efa7..318a98ef 100644
--- a/include/sys/socket.h
+++ b/include/sys/socket.h
@@ -350,6 +350,12 @@  int setsockopt (int, int, int, const void *, socklen_t);
 
 int sockatmark (int);
 
+#if _REDIR_TIME64
+#ifdef _GNU_SOURCE
+__REDIR(recvmmsg, __recvmmsg_time64);
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/stat.h b/include/sys/stat.h
index 9d096624..10d446c4 100644
--- a/include/sys/stat.h
+++ b/include/sys/stat.h
@@ -110,6 +110,15 @@  int lchmod(const char *, mode_t);
 #define off64_t off_t
 #endif
 
+#if _REDIR_TIME64
+__REDIR(stat, __stat_time64);
+__REDIR(fstat, __fstat_time64);
+__REDIR(lstat, __lstat_time64);
+__REDIR(fstatat, __fstatat_time64);
+__REDIR(futimens, __futimens_time64);
+__REDIR(utimensat, __utimensat_time64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/time.h b/include/sys/time.h
index c5cab814..cdc67ef6 100644
--- a/include/sys/time.h
+++ b/include/sys/time.h
@@ -56,6 +56,20 @@  int adjtime (const struct timeval *, struct timeval *);
 	(void)0 )
 #endif
 
+#if _REDIR_TIME64
+__REDIR(gettimeofday, __gettimeofday_time64);
+__REDIR(getitimer, __getitimer_time64);
+__REDIR(setitimer, __setitimer_time64);
+__REDIR(utimes, __utimes_time64);
+#if defined(_GNU_SOURCE) || defined(_BSD_SOURCE)
+__REDIR(futimes, __futimes_time64);
+__REDIR(futimesat, __futimesat_time64);
+__REDIR(lutimes, __lutimes_time64);
+__REDIR(settimeofday, __settimeofday_time64);
+__REDIR(adjtime, __adjtime64);
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/timeb.h b/include/sys/timeb.h
index 108c1f5c..628239b7 100644
--- a/include/sys/timeb.h
+++ b/include/sys/timeb.h
@@ -4,6 +4,8 @@ 
 extern "C" {
 #endif
 
+#include <features.h>
+
 #define __NEED_time_t
 
 #include <bits/alltypes.h>
@@ -16,6 +18,10 @@  struct timeb {
 
 int ftime(struct timeb *);
 
+#if _REDIR_TIME64
+__REDIR(ftime, __ftime64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/timerfd.h b/include/sys/timerfd.h
index 2794d36a..1b832cdd 100644
--- a/include/sys/timerfd.h
+++ b/include/sys/timerfd.h
@@ -20,6 +20,11 @@  int timerfd_create(int, int);
 int timerfd_settime(int, int, const struct itimerspec *, struct itimerspec *);
 int timerfd_gettime(int, struct itimerspec *);
 
+#if _REDIR_TIME64
+__REDIR(timerfd_settime, __timerfd_settime64);
+__REDIR(timerfd_gettime, __timerfd_gettime64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/timex.h b/include/sys/timex.h
index 2e688880..8b417e1b 100644
--- a/include/sys/timex.h
+++ b/include/sys/timex.h
@@ -91,6 +91,11 @@  struct timex {
 int adjtimex(struct timex *);
 int clock_adjtime(clockid_t, struct timex *);
 
+#if _REDIR_TIME64
+__REDIR(adjtimex, __adjtimex_time64);
+__REDIR(clock_adjtime, __clock_adjtime64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/sys/wait.h b/include/sys/wait.h
index 50c5c709..d9adbdec 100644
--- a/include/sys/wait.h
+++ b/include/sys/wait.h
@@ -53,6 +53,13 @@  pid_t wait4 (pid_t, int *, int, struct rusage *);
 #define WIFSIGNALED(s) (((s)&0xffff)-1U < 0xffu)
 #define WIFCONTINUED(s) ((s) == 0xffff)
 
+#if _REDIR_TIME64
+#if defined(_GNU_SOURCE) || defined(_BSD_SOURCE)
+__REDIR(wait3, __wait3_time64);
+__REDIR(wait4, __wait4_time64);
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/threads.h b/include/threads.h
index 8122b3b1..52ec3100 100644
--- a/include/threads.h
+++ b/include/threads.h
@@ -80,6 +80,12 @@  void tss_delete(tss_t);
 int tss_set(tss_t, void *);
 void *tss_get(tss_t);
 
+#if _REDIR_TIME64
+__REDIR(thrd_sleep, __thrd_sleep_time64);
+__REDIR(mtx_timedlock, __mtx_timedlock_time64);
+__REDIR(cnd_timedwait, __cnd_timedwait_time64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/time.h b/include/time.h
index 672b3fc3..5494df18 100644
--- a/include/time.h
+++ b/include/time.h
@@ -130,6 +130,34 @@  int stime(const time_t *);
 time_t timegm(struct tm *);
 #endif
 
+#if _REDIR_TIME64
+__REDIR(time, __time64);
+__REDIR(difftime, __difftime64);
+__REDIR(mktime, __mktime64);
+__REDIR(gmtime, __gmtime64);
+__REDIR(localtime, __localtime64);
+__REDIR(ctime, __ctime64);
+__REDIR(timespec_get, __timespec_get_time64);
+#if defined(_POSIX_SOURCE) || defined(_POSIX_C_SOURCE) \
+ || defined(_XOPEN_SOURCE) || defined(_GNU_SOURCE) \
+ || defined(_BSD_SOURCE)
+__REDIR(gmtime_r, __gmtime64_r);
+__REDIR(localtime_r, __localtime64_r);
+__REDIR(ctime_r, __ctime64_r);
+__REDIR(nanosleep, __nanosleep_time64);
+__REDIR(clock_getres, __clock_getres_time64);
+__REDIR(clock_gettime, __clock_gettime64);
+__REDIR(clock_settime, __clock_settime64);
+__REDIR(clock_nanosleep, __clock_nanosleep_time64);
+__REDIR(timer_settime, __timer_settime64);
+__REDIR(timer_gettime, __timer_gettime64);
+#endif
+#if defined(_GNU_SOURCE) || defined(_BSD_SOURCE)
+__REDIR(stime, __stime64);
+__REDIR(timegm, __timegm_time64);
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/utime.h b/include/utime.h
index dd5ff927..5755bd53 100644
--- a/include/utime.h
+++ b/include/utime.h
@@ -5,6 +5,8 @@ 
 extern "C" {
 #endif
 
+#include <features.h>
+
 #define __NEED_time_t
 
 #include <bits/alltypes.h>
@@ -16,6 +18,10 @@  struct utimbuf {
 
 int utime (const char *, const struct utimbuf *);
 
+#if _REDIR_TIME64
+__REDIR(utime, __utime64);
+#endif
+
 #ifdef __cplusplus
 }
 #endif

Comments

Rich Felker Oct. 21, 2019, 12:43 p.m.
On Sun, Oct 20, 2019 at 10:46:43PM -0400, Rich Felker wrote:
> >From 31aae539d35550a3db0641f25d16968ed6a0702c Mon Sep 17 00:00:00 2001
> From: Rich Felker <dalias@aerifal.cx>
> Date: Sun, 20 Oct 2019 19:24:53 -0400
> Subject: [PATCH 17/17] switch powerpc to 64-bit time_t
> 
> ---
>  arch/powerpc/arch.mak           |  1 +
>  arch/powerpc/bits/alltypes.h.in |  5 +++--
>  arch/powerpc/bits/ipcstat.h     |  2 +-
>  arch/powerpc/bits/msg.h         | 15 +++++++++------
>  arch/powerpc/bits/sem.h         | 10 ++++++----
>  arch/powerpc/bits/shm.h         | 16 +++++++++-------
>  arch/powerpc/bits/stat.h        |  6 +++++-
>  7 files changed, 34 insertions(+), 21 deletions(-)
>  create mode 100644 arch/powerpc/arch.mak
> 
> diff --git a/arch/powerpc/arch.mak b/arch/powerpc/arch.mak
> new file mode 100644
> index 00000000..aa4d05ce
> --- /dev/null
> +++ b/arch/powerpc/arch.mak
> @@ -0,0 +1 @@
> +COMPAT_SRC_DIRS = compat/time32
> diff --git a/arch/powerpc/bits/alltypes.h.in b/arch/powerpc/bits/alltypes.h.in
> index fd0c816c..8e003545 100644
> --- a/arch/powerpc/bits/alltypes.h.in
> +++ b/arch/powerpc/bits/alltypes.h.in
> @@ -1,3 +1,4 @@
> +#define _REDIR_TIME64 1
>  #define _Addr int
>  #define _Int64 long long
>  #define _Reg int
> @@ -18,5 +19,5 @@ TYPEDEF double double_t;
>  
>  TYPEDEF struct { long long __ll; long double __ld; } max_align_t;
>  
> -TYPEDEF long time_t;
> -TYPEDEF long suseconds_t;
> +TYPEDEF long long time_t;
> +TYPEDEF long long suseconds_t;
> diff --git a/arch/powerpc/bits/ipcstat.h b/arch/powerpc/bits/ipcstat.h
> index 0018ad1e..4f4fcb0c 100644
> --- a/arch/powerpc/bits/ipcstat.h
> +++ b/arch/powerpc/bits/ipcstat.h
> @@ -1 +1 @@
> -#define IPC_STAT 2
> +#define IPC_STAT 0x102

This could actually be dropped for powerpc...

> diff --git a/arch/powerpc/bits/msg.h b/arch/powerpc/bits/msg.h
> index 171c11a3..9fb15dcc 100644
> --- a/arch/powerpc/bits/msg.h
> +++ b/arch/powerpc/bits/msg.h
> @@ -1,15 +1,18 @@
>  struct msqid_ds {
>  	struct ipc_perm msg_perm;
> -	int __unused1;
> -	time_t msg_stime;
> -	int __unused2;
> -	time_t msg_rtime;
> -	int __unused3;
> -	time_t msg_ctime;
> +	unsigned long __msg_stime_hi;
> +	unsigned long __msg_stime_lo;
> +	unsigned long __msg_rtime_hi;
> +	unsigned long __msg_rtime_lo;
> +	unsigned long __msg_ctime_hi;
> +	unsigned long __msg_ctime_lo;

by making this just:

> +	time_t msg_stime;
> +	time_t msg_rtime;
> +	time_t msg_ctime;

since the alignments and endianness are correct (I'm pretty sure
they're correct for all 3 structs). Any ppc folks want to confirm
that?

It's not a big deal either way but would be slightly "nicer".

Rich
Rich Felker Oct. 27, 2019, 4:15 a.m.
On Mon, Oct 21, 2019 at 08:43:35AM -0400, Rich Felker wrote:
> On Sun, Oct 20, 2019 at 10:46:43PM -0400, Rich Felker wrote:
> > >From 31aae539d35550a3db0641f25d16968ed6a0702c Mon Sep 17 00:00:00 2001
> > From: Rich Felker <dalias@aerifal.cx>
> > Date: Sun, 20 Oct 2019 19:24:53 -0400
> > Subject: [PATCH 17/17] switch powerpc to 64-bit time_t
> > 
> > ---
> >  arch/powerpc/arch.mak           |  1 +
> >  arch/powerpc/bits/alltypes.h.in |  5 +++--
> >  arch/powerpc/bits/ipcstat.h     |  2 +-
> >  arch/powerpc/bits/msg.h         | 15 +++++++++------
> >  arch/powerpc/bits/sem.h         | 10 ++++++----
> >  arch/powerpc/bits/shm.h         | 16 +++++++++-------
> >  arch/powerpc/bits/stat.h        |  6 +++++-
> >  7 files changed, 34 insertions(+), 21 deletions(-)
> >  create mode 100644 arch/powerpc/arch.mak
> > 
> > diff --git a/arch/powerpc/arch.mak b/arch/powerpc/arch.mak
> > new file mode 100644
> > index 00000000..aa4d05ce
> > --- /dev/null
> > +++ b/arch/powerpc/arch.mak
> > @@ -0,0 +1 @@
> > +COMPAT_SRC_DIRS = compat/time32
> > diff --git a/arch/powerpc/bits/alltypes.h.in b/arch/powerpc/bits/alltypes.h.in
> > index fd0c816c..8e003545 100644
> > --- a/arch/powerpc/bits/alltypes.h.in
> > +++ b/arch/powerpc/bits/alltypes.h.in
> > @@ -1,3 +1,4 @@
> > +#define _REDIR_TIME64 1
> >  #define _Addr int
> >  #define _Int64 long long
> >  #define _Reg int
> > @@ -18,5 +19,5 @@ TYPEDEF double double_t;
> >  
> >  TYPEDEF struct { long long __ll; long double __ld; } max_align_t;
> >  
> > -TYPEDEF long time_t;
> > -TYPEDEF long suseconds_t;
> > +TYPEDEF long long time_t;
> > +TYPEDEF long long suseconds_t;
> > diff --git a/arch/powerpc/bits/ipcstat.h b/arch/powerpc/bits/ipcstat.h
> > index 0018ad1e..4f4fcb0c 100644
> > --- a/arch/powerpc/bits/ipcstat.h
> > +++ b/arch/powerpc/bits/ipcstat.h
> > @@ -1 +1 @@
> > -#define IPC_STAT 2
> > +#define IPC_STAT 0x102
> 
> This could actually be dropped for powerpc...
> 
> > diff --git a/arch/powerpc/bits/msg.h b/arch/powerpc/bits/msg.h
> > index 171c11a3..9fb15dcc 100644
> > --- a/arch/powerpc/bits/msg.h
> > +++ b/arch/powerpc/bits/msg.h
> > @@ -1,15 +1,18 @@
> >  struct msqid_ds {
> >  	struct ipc_perm msg_perm;
> > -	int __unused1;
> > -	time_t msg_stime;
> > -	int __unused2;
> > -	time_t msg_rtime;
> > -	int __unused3;
> > -	time_t msg_ctime;
> > +	unsigned long __msg_stime_hi;
> > +	unsigned long __msg_stime_lo;
> > +	unsigned long __msg_rtime_hi;
> > +	unsigned long __msg_rtime_lo;
> > +	unsigned long __msg_ctime_hi;
> > +	unsigned long __msg_ctime_lo;
> 
> by making this just:
> 
> > +	time_t msg_stime;
> > +	time_t msg_rtime;
> > +	time_t msg_ctime;
> 
> since the alignments and endianness are correct (I'm pretty sure
> they're correct for all 3 structs). Any ppc folks want to confirm
> that?
> 
> It's not a big deal either way but would be slightly "nicer".

If there's any possibility of having a little-endian 32-bit powerpc
target, then I think it's probably best not to do this -- it would
have to be backed out conditional on endianness.

Rich
Rich Felker Oct. 27, 2019, 4:26 a.m.
On Sun, Oct 20, 2019 at 10:46:43PM -0400, Rich Felker wrote:
> demonstrably working). The one omission I'm aware of is what to do
> with struct utmpx, which is not actually used at present in any libc
> interfaces and thus not part of the ABI surface of libc. That will be
> addressed in a separate thread.

Or here. So, the story on utmpx: we can either

1. match the current size on 32-bit archs, but move the timeval to
   unused space at the end where a time64 version fits, or

2. match the current size and layout of the 64-bit struct, making it
   possible to share records between 32- and 64-bit processes on the
   same machine.

Keep in mind that this struct is not used anywhere in libc presently,
but normally it's used as a format for on-disk records.

I'm kinda leaning towards option 2, but being that I don't use (and
hate) utmp, I'd rather hear opinions from people who do use it. Either
way time fields in existing data will break, so it's a question of
whether that one-time breakage is already sufficient to go a bit
further and get 32/64 compat afterwards.

Rich
Laurent Bercot Oct. 27, 2019, 8:32 a.m.
>Or here. So, the story on utmpx: we can either
>
>1. match the current size on 32-bit archs, but move the timeval to
>    unused space at the end where a time64 version fits, or
>
>2. match the current size and layout of the 64-bit struct, making it
>    possible to share records between 32- and 64-bit processes on the
>    same machine.
>
>Keep in mind that this struct is not used anywhere in libc presently,
>but normally it's used as a format for on-disk records.
>
>I'm kinda leaning towards option 2, but being that I don't use (and
>hate) utmp, I'd rather hear opinions from people who do use it. Either
>way time fields in existing data will break, so it's a question of
>whether that one-time breakage is already sufficient to go a bit
>further and get 32/64 compat afterwards.

I don't use the libc's utmpx, but I maintain utmps, which is a secure 
implementation of utmp, including the definition of struct utmpx.
I haven't been following the time64 thing closely. The current struct
utmpx definition includes a struct timeval. Will it need to change,
or will musl's struct timeval change be enough and naturally propagate
so the struct utmpx will become time64-compatible?

On-disk data is not a problem. On the distro that I know uses utmps
(Adélie), the utmp/wtmp records, by design, do not survive a reboot,
so a reboot will fix everything - and will be mandatory anyway on
arches where the musl ABI changes.

I'm not aware of any distribution that uses musl, doesn't use utmps,
and still keeps on-disk utmpx records.

--
Laurent
Rich Felker Oct. 27, 2019, 2:53 p.m.
On Sun, Oct 27, 2019 at 08:32:57AM +0000, Laurent Bercot wrote:
> >Or here. So, the story on utmpx: we can either
> >
> >1. match the current size on 32-bit archs, but move the timeval to
> >   unused space at the end where a time64 version fits, or
> >
> >2. match the current size and layout of the 64-bit struct, making it
> >   possible to share records between 32- and 64-bit processes on the
> >   same machine.
> >
> >Keep in mind that this struct is not used anywhere in libc presently,
> >but normally it's used as a format for on-disk records.
> >
> >I'm kinda leaning towards option 2, but being that I don't use (and
> >hate) utmp, I'd rather hear opinions from people who do use it. Either
> >way time fields in existing data will break, so it's a question of
> >whether that one-time breakage is already sufficient to go a bit
> >further and get 32/64 compat afterwards.
> 
> I don't use the libc's utmpx, but I maintain utmps, which is a
> secure implementation of utmp, including the definition of struct
> utmpx.
> I haven't been following the time64 thing closely. The current struct
> utmpx definition includes a struct timeval. Will it need to change,
> or will musl's struct timeval change be enough and naturally propagate
> so the struct utmpx will become time64-compatible?

It will naturally propagate even if nothing is done, but then you have
the worst of both worlds of 1 and 2. You neither maintain the size and
layout of other members (which would be useful if you have old data or
a mix of old and new binaries using it) nor gain any useful
compatibility (between 32- and 64-bit on same system). You just get a
new size and layout that doesn't match either. So it's better to make
some change, I think.

Note that the difference between "do nothing" and option 2 is
basically nothing except putting padding around ut_session so that
ut_tv will start on a 0 mod 8 boundary. This will happen naturally on
most archs but not i386 (and m68k but there's no corresponding 64-bit
arch there anyway). The proposal I have in mind is basically:

-	long ut_session;
+#if __BYTE_ORDER == 1234
+	int ut_session, __ut_pad2;
+#else
+	int __ut_pad2, ut_session;
+#endif

doing this for 64-bit too since ut_session is semantically 32-bit
(pid_t) and glibc has it as 32-bit on x86_64. (I'd also add explicit
padding for ut_type just because m68k is wacky and doesn't align ints
even, to fix that while we have the chance.)

> On-disk data is not a problem. On the distro that I know uses utmps
> (Adélie), the utmp/wtmp records, by design, do not survive a reboot,
> so a reboot will fix everything - and will be mandatory anyway on
> arches where the musl ABI changes.

Reboot is not mandatory; as usual, just atomic replacement of libc.so
is.

> I'm not aware of any distribution that uses musl, doesn't use utmps,
> and still keeps on-disk utmpx records.

Thanks.

Rich
Matias Fonzo Oct. 27, 2019, 8:12 p.m.
Hello Laurent,

Can utmps work without s6?.  I mean, independently of the init system or 
distribution...

El 2019-10-27 05:32, Laurent Bercot escribió:
>> Or here. So, the story on utmpx: we can either
>> 
>> 1. match the current size on 32-bit archs, but move the timeval to
>>    unused space at the end where a time64 version fits, or
>> 
>> 2. match the current size and layout of the 64-bit struct, making it
>>    possible to share records between 32- and 64-bit processes on the
>>    same machine.
>> 
>> Keep in mind that this struct is not used anywhere in libc presently,
>> but normally it's used as a format for on-disk records.
>> 
>> I'm kinda leaning towards option 2, but being that I don't use (and
>> hate) utmp, I'd rather hear opinions from people who do use it. Either
>> way time fields in existing data will break, so it's a question of
>> whether that one-time breakage is already sufficient to go a bit
>> further and get 32/64 compat afterwards.
> 
> I don't use the libc's utmpx, but I maintain utmps, which is a secure
> implementation of utmp, including the definition of struct utmpx.
> I haven't been following the time64 thing closely. The current struct
> utmpx definition includes a struct timeval. Will it need to change,
> or will musl's struct timeval change be enough and naturally propagate
> so the struct utmpx will become time64-compatible?
> 
> On-disk data is not a problem. On the distro that I know uses utmps
> (Adélie), the utmp/wtmp records, by design, do not survive a reboot,
> so a reboot will fix everything - and will be mandatory anyway on
> arches where the musl ABI changes.
> 
> I'm not aware of any distribution that uses musl, doesn't use utmps,
> and still keeps on-disk utmpx records.
> 
> --
> Laurent
Rich Felker Oct. 27, 2019, 9:14 p.m.
On Sun, Oct 27, 2019 at 05:12:59PM -0300, Matias Fonzo wrote:
> Hello Laurent,
> 
> Can utmps work without s6?.  I mean, independently of the init
> system or distribution...

Laurent could answer in better detail, but as a quick answer, there's
no requirement from having s6 installed that you use it as your init
system.

Rich


> El 2019-10-27 05:32, Laurent Bercot escribió:
> >>Or here. So, the story on utmpx: we can either
> >>
> >>1. match the current size on 32-bit archs, but move the timeval to
> >>   unused space at the end where a time64 version fits, or
> >>
> >>2. match the current size and layout of the 64-bit struct, making it
> >>   possible to share records between 32- and 64-bit processes on the
> >>   same machine.
> >>
> >>Keep in mind that this struct is not used anywhere in libc presently,
> >>but normally it's used as a format for on-disk records.
> >>
> >>I'm kinda leaning towards option 2, but being that I don't use (and
> >>hate) utmp, I'd rather hear opinions from people who do use it. Either
> >>way time fields in existing data will break, so it's a question of
> >>whether that one-time breakage is already sufficient to go a bit
> >>further and get 32/64 compat afterwards.
> >
> >I don't use the libc's utmpx, but I maintain utmps, which is a secure
> >implementation of utmp, including the definition of struct utmpx.
> >I haven't been following the time64 thing closely. The current struct
> >utmpx definition includes a struct timeval. Will it need to change,
> >or will musl's struct timeval change be enough and naturally propagate
> >so the struct utmpx will become time64-compatible?
> >
> >On-disk data is not a problem. On the distro that I know uses utmps
> >(Adélie), the utmp/wtmp records, by design, do not survive a reboot,
> >so a reboot will fix everything - and will be mandatory anyway on
> >arches where the musl ABI changes.
> >
> >I'm not aware of any distribution that uses musl, doesn't use utmps,
> >and still keeps on-disk utmpx records.
> >
> >--
> >Laurent
Matias Fonzo Oct. 27, 2019, 9:53 p.m.
Hello Rich,

El 2019-10-27 18:14, Rich Felker escribió:
> On Sun, Oct 27, 2019 at 05:12:59PM -0300, Matias Fonzo wrote:
>> Hello Laurent,
>> 
>> Can utmps work without s6?.  I mean, independently of the init
>> system or distribution...
> 
> Laurent could answer in better detail, but as a quick answer, there's
> no requirement from having s6 installed that you use it as your init
> system.
> 

I thought so, but there could be a configuration-side requirement or 
daemon (setup) to work properly, I don't know.  We are using perp for 
Dragora.

> 
> 
>> El 2019-10-27 05:32, Laurent Bercot escribió:
>> >>Or here. So, the story on utmpx: we can either
>> >>
>> >>1. match the current size on 32-bit archs, but move the timeval to
>> >>   unused space at the end where a time64 version fits, or
>> >>
>> >>2. match the current size and layout of the 64-bit struct, making it
>> >>   possible to share records between 32- and 64-bit processes on the
>> >>   same machine.
>> >>
>> >>Keep in mind that this struct is not used anywhere in libc presently,
>> >>but normally it's used as a format for on-disk records.
>> >>
>> >>I'm kinda leaning towards option 2, but being that I don't use (and
>> >>hate) utmp, I'd rather hear opinions from people who do use it. Either
>> >>way time fields in existing data will break, so it's a question of
>> >>whether that one-time breakage is already sufficient to go a bit
>> >>further and get 32/64 compat afterwards.
>> >
>> >I don't use the libc's utmpx, but I maintain utmps, which is a secure
>> >implementation of utmp, including the definition of struct utmpx.
>> >I haven't been following the time64 thing closely. The current struct
>> >utmpx definition includes a struct timeval. Will it need to change,
>> >or will musl's struct timeval change be enough and naturally propagate
>> >so the struct utmpx will become time64-compatible?
>> >
>> >On-disk data is not a problem. On the distro that I know uses utmps
>> >(Adélie), the utmp/wtmp records, by design, do not survive a reboot,
>> >so a reboot will fix everything - and will be mandatory anyway on
>> >arches where the musl ABI changes.
>> >
>> >I'm not aware of any distribution that uses musl, doesn't use utmps,
>> >and still keeps on-disk utmpx records.
>> >
>> >--
>> >Laurent
Laurent Bercot Oct. 27, 2019, 11:27 p.m.
Hi Matias,

  There is a run-time requirement for s6, but it's not an absolute one:
the utmps-utmpd and utmps-wtmpd programs simply rely on an interface
provided by s6-ipcserver(d). If you can provide the same interface,
you can do without s6.

  utmps-utmpd and utmps-wtmpd expect:
  - to be launched via an inetd-like listening on the configured Unix
domain socket, with stdin reading from the client and stdout writing
to the client.
  - some environment variables:
    * PROTO must be set to IPC.
    * IPCREMOTEEUID must be set to the effective uid of the client.
    * IPCREMOTEEGID must be set to the effective gid of the client.
    Those last two are obtained on Linux via a struct ucred and the
SO_PEERCRED option to getsockopt(). You can't fake that, it's the
very reason why utmps is secure.

  Of course, you could also package s6 in Dragora. If you already have
a perp supervision tree, you don't even have to run a s6 one. On the
other hand, that's a risky proposition, because you might end up liking
it and wanting to use it more. %-)

--
  Laurent
Matias Fonzo Oct. 28, 2019, 9:31 p.m.
Hi Laurent,

Thanks for the explanation and to give more details about it.  :-)

El 2019-10-27 20:27, Laurent Bercot escribió:
> 
>  There is a run-time requirement for s6, but it's not an absolute one:
> the utmps-utmpd and utmps-wtmpd programs simply rely on an interface
> provided by s6-ipcserver(d). If you can provide the same interface,
> you can do without s6.
> 
>  utmps-utmpd and utmps-wtmpd expect:
>  - to be launched via an inetd-like listening on the configured Unix
> domain socket, with stdin reading from the client and stdout writing
> to the client.
>  - some environment variables:
>    * PROTO must be set to IPC.
>    * IPCREMOTEEUID must be set to the effective uid of the client.
>    * IPCREMOTEEGID must be set to the effective gid of the client.
>    Those last two are obtained on Linux via a struct ucred and the
> SO_PEERCRED option to getsockopt(). You can't fake that, it's the
> very reason why utmps is secure.
> 
>  Of course, you could also package s6 in Dragora. If you already have
> a perp supervision tree, you don't even have to run a s6 one. On the
> other hand, that's a risky proposition, because you might end up liking
> it and wanting to use it more. %-)
> 
> --
>  Laurent
Rich Felker Oct. 28, 2019, 10:22 p.m.
On Sun, Oct 27, 2019 at 12:26:45AM -0400, Rich Felker wrote:
> On Sun, Oct 20, 2019 at 10:46:43PM -0400, Rich Felker wrote:
> > demonstrably working). The one omission I'm aware of is what to do
> > with struct utmpx, which is not actually used at present in any libc
> > interfaces and thus not part of the ABI surface of libc. That will be
> > addressed in a separate thread.
> 
> Or here. So, the story on utmpx: we can either
> 
> 1. match the current size on 32-bit archs, but move the timeval to
>    unused space at the end where a time64 version fits, or
> 
> 2. match the current size and layout of the 64-bit struct, making it
>    possible to share records between 32- and 64-bit processes on the
>    same machine.
> 
> Keep in mind that this struct is not used anywhere in libc presently,
> but normally it's used as a format for on-disk records.
> 
> I'm kinda leaning towards option 2, but being that I don't use (and
> hate) utmp, I'd rather hear opinions from people who do use it. Either
> way time fields in existing data will break, so it's a question of
> whether that one-time breakage is already sufficient to go a bit
> further and get 32/64 compat afterwards.

Thanks to git://repo.or.cz/musl-tools.git abi checker, I found a few
other minor details that have been overlooked. In addition to utmp[x],
there's also struct lastlog which depends on time_t. For it there's
really no choice to be made; change necessarily breaks size/layout of
the type, and the change that falls out naturally from time_t changing
is the "right" one (matches 64-bit struct too).

There's also ntptimeval, which I'd mentioned at one point before but
forgotten about. It's not used anywhere in libc and dubious that we
even have the type defined at all. So changes in it are no big deal.

Finally, prstatus_t (struct elf_prstatus) has timeval members, and as
I understand it this structure is ABI for core files or something. I
can't find any code that's actually using it though. This structure
can't maintain both ABI and API, so presumably API must break, i.e.
the types of the members must change. Or we just remove it entirely.
I'd be rather happy if we could do that, actually. Can any distro
folks chime in on whether you've actually encountered users of this
cruft?

Rich
Rich Felker Oct. 29, 2019, 7:52 p.m.
On Sun, Oct 20, 2019 at 10:46:43PM -0400, Rich Felker wrote:
> The attached patch series on top of present git master (commit
> 9b2921bea1d5017832e1b45d1fd64220047a9802) should contain all changes
> needed for fully working time64 on 32-bit archs, in a form that's
> plausibly ready for commit (no makeshift hacks just to get things
> demonstrably working). The one omission I'm aware of is what to do
> with struct utmpx, which is not actually used at present in any libc
> interfaces and thus not part of the ABI surface of libc. That will be
> addressed in a separate thread.
> 
> Comments and basic testing are welcome at this point. It should be
> possible to build for any of the 32-bit archs, but I have only tested
> build for a few and only tested execution on i386 and sh.
> 
> Some useful checks for anyone wanting to help test, especially on the
> more obscure archs:
> 
> [...]
> 
> - Anything look odd about time-related types? timeval/timespec members
>   at positions that aren't naturally aligned?

I tested this on i386 (with low alignment requirement, only 4 bytes)
with the attached program and it seems like I indeed got the padding
slots as-intended on the ipc types. This should mean the adjacent
explicit-padding members have no effect on other archs using the same
changes but with natural alignment, which matches the intent. As usual
mips and ppc are gratuitously different and seemed right to me but
should perhaps be double-checked at some point. However since they
have natural alignment the worst that could happen is leaving extra
padding slots we don't need.

> [...]
> 
> - Does it work with clock set post-2038?

I tested this for arch/sh on j2 (fpga) which was the most convenient
place I had a 5.x kernel, and indeed it works!

> Being that only the final switchover patches are a big step to take,
> I'll probably start pushing the earlier patches in this series pretty
> soon, but want to give a little time for more eyes on them.

I'm going to push them all as a group very soon.

> Note that the final switchovers are split out by arch right now,
> mainly because that was the way that made sense to create them (sed to
> replace arch name, apply to next arch, fix rejects manually) but also
> because it helps compare/review them against each other. It would be
> possible to squash them together for final merge but I probably won't.

I've actually opted to squash them together just because it
facilitates writing a meaningful commit message that explains the ABI
properties, consequences, stability policy, mechanics of the change,
etc. in a non-redundant manner (the explanation is the same across all
archs). The diff naturally splits by arch just by the default pathname
sorting (or by applying the diff just to the appropriate dir under
arch/), so nothing is lost in terms of ability to look at changes for
individual archs separately or compare them.

Rich
Rich Felker Oct. 29, 2019, 7:53 p.m.
On Tue, Oct 29, 2019 at 03:52:45PM -0400, Rich Felker wrote:
> On Sun, Oct 20, 2019 at 10:46:43PM -0400, Rich Felker wrote:
> > The attached patch series on top of present git master (commit
> > 9b2921bea1d5017832e1b45d1fd64220047a9802) should contain all changes
> > needed for fully working time64 on 32-bit archs, in a form that's
> > plausibly ready for commit (no makeshift hacks just to get things
> > demonstrably working). The one omission I'm aware of is what to do
> > with struct utmpx, which is not actually used at present in any libc
> > interfaces and thus not part of the ABI surface of libc. That will be
> > addressed in a separate thread.
> > 
> > Comments and basic testing are welcome at this point. It should be
> > possible to build for any of the 32-bit archs, but I have only tested
> > build for a few and only tested execution on i386 and sh.
> > 
> > Some useful checks for anyone wanting to help test, especially on the
> > more obscure archs:
> > 
> > [...]
> > 
> > - Anything look odd about time-related types? timeval/timespec members
> >   at positions that aren't naturally aligned?
> 
> I tested this on i386 (with low alignment requirement, only 4 bytes)
> with the attached program and it seems like I indeed got the padding
> slots as-intended on the ipc types. This should mean the adjacent
> explicit-padding members have no effect on other archs using the same
> changes but with natural alignment, which matches the intent. As usual
> mips and ppc are gratuitously different and seemed right to me but
> should perhaps be double-checked at some point. However since they
> have natural alignment the worst that could happen is leaving extra
> padding slots we don't need.

Attachment forgotten, here. Usage is to build against the appropriate
libc/headers and save output, then diff outputs as desired.

Rich
#include <stddef.h>
#include <stdio.h>
#include <sys/sem.h>
#include <sys/shm.h>
#include <sys/msg.h>
#include <sys/stat.h>

#define O(s,m) printf("offsetof(%s, %s)==%zd\n", #s, #m, offsetof(s,m))

int main()
{
	O(struct semid_ds, sem_perm);
	O(struct semid_ds, sem_otime);
	O(struct semid_ds, sem_ctime);
	O(struct semid_ds, sem_nsems);

	O(struct shmid_ds, shm_perm);
	O(struct shmid_ds, shm_segsz);
	O(struct shmid_ds, shm_atime);
	O(struct shmid_ds, shm_dtime);
	O(struct shmid_ds, shm_ctime);
	O(struct shmid_ds, shm_cpid);
	O(struct shmid_ds, shm_lpid);
	O(struct shmid_ds, shm_nattch);

	O(struct msqid_ds, msg_perm);
	O(struct msqid_ds, msg_stime);
	O(struct msqid_ds, msg_rtime);
	O(struct msqid_ds, msg_ctime);
	O(struct msqid_ds, msg_cbytes);
	O(struct msqid_ds, msg_qnum);
	O(struct msqid_ds, msg_qbytes);
	O(struct msqid_ds, msg_lspid);
	O(struct msqid_ds, msg_lrpid);

	O(struct stat, st_dev);
	O(struct stat, st_ino);
	O(struct stat, st_mode);
	O(struct stat, st_nlink);
	O(struct stat, st_uid);
	O(struct stat, st_gid);
	O(struct stat, st_rdev);
	O(struct stat, st_size);
	O(struct stat, st_blksize);
	O(struct stat, st_blocks);
	O(struct stat, st_atim);
	O(struct stat, st_mtim);
	O(struct stat, st_ctim);
}
Rich Felker Oct. 29, 2019, 11:08 p.m.
On Sun, Oct 20, 2019 at 10:46:43PM -0400, Rich Felker wrote:
> The attached patch series on top of present git master (commit
> 9b2921bea1d5017832e1b45d1fd64220047a9802) should contain all changes
> needed for fully working time64 on 32-bit archs, in a form that's
> plausibly ready for commit (no makeshift hacks just to get things
> demonstrably working). The one omission I'm aware of is what to do
> with struct utmpx, which is not actually used at present in any libc
> interfaces and thus not part of the ABI surface of libc. That will be
> addressed in a separate thread.
> 
> Comments and basic testing are welcome at this point. It should be
> possible to build for any of the 32-bit archs, but I have only tested
> build for a few and only tested execution on i386 and sh.
> 
> Some useful checks for anyone wanting to help test, especially on the
> more obscure archs:
> 
> [...]
> 
> - Does software built against new libc headers basically work?

strace breaks at build time with static_assert that IPC_STAT==2. This
had me momentarily doubting the whole approach of redefining it rather
than making __semctl_time64, etc., but it turns out strace also
depends on struct semid_ds, shmid_ds, and msqid_ds matching the kernel
layouts, which is impossible with time64, so there's really a
fundamental problem here.

Fortunately, strace already had the right code and just wasn't using
it: if HAVE_SYS_SEM_H, etc. aren't defined in config.h (from
autoconf), it falls back to using linux/sem.h, etc., which are exactly
what it should have been using all along: the kernel struct layouts
and command macro definitions.

I think the right patch to strace will be simply removing use of libc
sysvipc headers (always using the kernel ones); it doesn't seem
practical to write a configure test to determine that the libc ones
are usable for parsing the kernel struct.

So, first "oh no!" averted! But this may be a preview of what's to
come. (And I might have chosen strace as an early test case because I
expected things like this.)

Rich