Bryan

Bryan

twitter
medium

Do you really understand time.Now()?

This article analyzes the source code of Go1.20.4, and there may be differences in higher or lower versions.

Overview: time.Time#

Without further ado, let's look at the source code.

// To reduce the length of the article and highlight the key points, some comments have been edited.


// A Time represents an instant in time with nanosecond precision.
// 
// The zero value of type Time is January 1, year 1, 00:00:00.000000000 UTC.
//
// In addition to the required “wall clock” reading, a Time may contain an optional
// reading of the current process's monotonic clock, to provide additional precision
// for comparison or subtraction.
type Time struct {
	// wall and ext encode the wall time seconds, wall time nanoseconds,
	// and optional monotonic clock reading in nanoseconds.
	//
	// From high to low bit position, wall encodes a 1-bit flag (hasMonotonic),
	// a 33-bit seconds field, and a 30-bit wall time nanoseconds field.
	// The nanoseconds field is in the range [0, 999999999].
	// If the hasMonotonic bit is 0, then the 33-bit field must be zero
	// and the full signed 64-bit wall seconds since Jan 1 year 1 is stored in ext.
	// If the hasMonotonic bit is 1, then the 33-bit field holds a 33-bit
	// unsigned wall seconds since Jan 1 year 1885, and ext holds a
	// signed 64-bit monotonic clock reading, nanoseconds since process start.
	wall uint64
	ext  int64

	// The focus of this article is not on time zones, or a separate article will discuss related topics.
	loc *Location
}

When discussing the concept of "time," or more precisely, an instant, we usually have no doubts. However, unlike the unique time points in our lives, modern computers actually have two types of clocks: calendar clock (time-of-day clock / wall clock) and monotonic clock.

The time we usually see (including timestamps, displays of year, month, day, hour, minute, second, etc.) is generally from the calendar clock. However, when we want to calculate time intervals, the difference in calendar clocks can be negative—time can be "adjusted" between two calculations. Therefore, we need a method to stably obtain the elapsed time between two time points, which is the origin of the monotonic clock. The name "monotonic clock" comes from its monotonically increasing characteristic; it is usually not a real time value, and considering the value of the monotonic clock is meaningless; its sole purpose is to provide stable calculations of time differences.

Further discussion of calendar clocks and monotonic clocks exceeds the scope of this article. I recommend reading the section on unreliable clocks in Chapter 8 of "Designing Data-Intensive Applications". But looking back at the source code of Time—it includes both the calendar clock and the monotonic clock.

The 64 bits of wall are divided into three parts: 1+33+30, where the first part (the highest bit) is named hasMonotonic, which determines how the calendar clock and monotonic clock are stored.

  • When hasMonotonic = 1
    • The second part of wall (33 bits) stores the number of seconds since January 1, 1885 (unsigned).
    • The third part of wall (30 bits) stores the nanosecond part (range [0, 10^9-1]).
    • ext stores the number of nanoseconds since the process started (signed).
  • When hasMonotonic = 0
    • The second part of wall (33 bits) is all 0.
    • The third part of wall (30 bits) stores the nanosecond part (range [0, 10^9-1]).
    • ext stores the number of seconds since January 1, year 1 (signed).

Here, we conduct some extreme analysis:

  1. The maximum decimal value of the nanosecond part 10^9-1 corresponds to 30 bits in binary, ensuring that the third part of wall does not overflow.
  2. The second part of wall (33 bits) corresponds to seconds, with a maximum value of 8589934591 seconds, approximately 272.4 years, usable until the year 2157 from January 1, 1885.
  3. The maximum value of signed 64-bit nanoseconds is approximately 292.47 years (it shouldn't be that a program runs continuously for that long).
    • In fact, according to Golang's internal implementation, the maximum limit is constrained by the system's returned monotonic clock; for Linux, the entire system uptime can reach a maximum of 292.47 years.
  4. The maximum value of signed 64-bit seconds since January 1, year 1 can reach 292.47 billion years, which we will not see overflow in our lifetime.

Getting Time#

time.Now()#

// Now returns the current local time.
func Now() Time {
	sec, nsec, mono := now()
	mono -= startNano
	sec += unixToInternal - minWall
	if uint64(sec)>>33 != 0 {
		// Seconds field overflowed the 33 bits available when
		// storing a monotonic time. This will be true after
		// March 16, 2157.
		return Time{uint64(nsec), sec + minWall, Local}
	}
	return Time{hasMonotonic | uint64(sec)<<nsecShift | uint64(nsec), mono, Local}
}

// Provided by package runtime.
func now() (sec int64, nsec int32, mono int64)

After gaining a sufficient understanding of time.Time, we can easily comprehend the logic designed in this code—preserving monotonic as much as possible.

The now method is used to return the current time, and its specific implementation depends on the system. Its three return values are:

  • sec - Unix timestamp (the number of seconds since January 1, 1970).
  • nsec - nanosecond offset [0, 10^9-1].
  • mono - system-level monotonic clock value.

First, there are two adjustments:

  • mono -= startNano

    • As mentioned earlier, the monotonic clock value stored in Time is not the system's returned value but the number of nanoseconds since the process started. Therefore, a subtraction operation is performed here (subtracting the system's monotonic clock value at the time of system startup).
    // Monotonic times are reported as offsets from startNano.
    // We initialize startNano to runtimeNano() - 1 so that on systems where
    // monotonic time resolution is fairly low (e.g. Windows 2008
    // which appears to have a default resolution of 15ms),
    // we avoid ever reporting a monotonic time of 0.
    // (Callers may want to use 0 as "time not set".)
    var startNano int64 = runtimeNano() - 1
    
  • sec += unixToInternal - minWall

    • In the case of hasMonotonic=1, the Time stores the number of seconds since January 1, 1885, while the system returns the number of seconds since January 1, 1970. Therefore, sec needs to be adjusted by subtracting the 85 years.
    const (
    	unixToInternal int64 = (1969*365 + 1969/4 - 1969/100 + 1969/400) * secondsPerDay
    )
    
    const (
    	wallToInternal int64 = (1884*365 + 1884/4 - 1884/100 + 1884/400) * secondsPerDay
    	minWall      = wallToInternal               // year 1885
    )
    

The subsequent judgment logic is straightforward: if possible, use hasMonotonic=1 for storage; otherwise (when the time does not fall within the range of 1885-2157), use hasMonotonic=0 for storage. The later judgment branches... those who can see this article probably won't encounter them.

time.Unix()#

// Unix returns the local Time corresponding to the given Unix time,
// sec seconds and nsec nanoseconds since January 1, 1970 UTC.
// It is valid to pass nsec outside the range [0, 999999999].
// Not all sec values have a corresponding time value. One such
// value is 1<<63-1 (the largest int64 value).
func Unix(sec int64, nsec int64) Time {
	if nsec < 0 || nsec >= 1e9 {
		n := nsec / 1e9
		sec += n
		nsec -= n * 1e9
		if nsec < 0 {
			nsec += 1e9
			sec--
		}
	}
	return unixTime(sec, int32(nsec))
}

func unixTime(sec int64, nsec int32) Time {
	return Time{uint64(nsec), sec + unixToInternal, Local}
}

const (
	unixToInternal int64 = (1969*365 + 1969/4 - 1969/100 + 1969/400) * secondsPerDay
)

Unix primarily does one thing: it uses the Unix timestamp (epoch) to generate a time with hasMonotonic=0.

The if logic before calling unixTime is to simplify the business code—if the nsec part is not within the limit range of [0, 10^9-1], it corrects it for us.

Oh, by the way, besides the most commonly used second-level timestamps, we may also encounter millisecond-level and microsecond-level timestamps, and Go also provides simplified calls for us:

// UnixMilli returns the local Time corresponding to the given Unix time,
// msec milliseconds since January 1, 1970 UTC.
func UnixMilli(msec int64) Time {
	return Unix(msec/1e3, (msec%1e3)*1e6)
}

// UnixMicro returns the local Time corresponding to the given Unix time,
// usec microseconds since January 1, 1970 UTC.
func UnixMicro(usec int64) Time {
	return Unix(usec/1e6, (usec%1e6)*1e3)
}

Hmm... if you ask me about nanosecond-level timestamps... just call time.Unix(0, nanoseconds) directly! 😂

time.Date()#

// To reduce the length of the article and highlight the key points, some code and comments have been edited.

// Date returns the Time corresponding to
//
//	yyyy-mm-dd hh:mm:ss + nsec nanoseconds
//
// The month, day, hour, min, sec, and nsec values may be outside
// their usual ranges and will be normalized during the conversion.
// For example, October 32 converts to November 1.
func Date(year int, month Month, day, hour, min, sec, nsec int, loc *Location) Time {
	...

	// Normalize overflow
	...

	// Compute days since the absolute epoch.
	d := daysSinceEpoch(year)

	// Add in days before this month.
	d += uint64(daysBefore[month-1])
	if isLeap(year) && month >= March {
		d++ // February 29
	}

	// Add in days before today.
	d += uint64(day - 1)

	// Add in time elapsed today.
	abs := d * secondsPerDay
	abs += uint64(hour*secondsPerHour + min*secondsPerMinute + sec)

	unix := int64(abs) + (absoluteToInternal + internalToUnix)

	// Look for zone offset for expected time, so we can adjust to UTC.
	...

	t := unixTime(unix, int32(nsec))
	t.setLoc(loc)
	return t
}

Most of the logic has been directly omitted. In simple terms, Date does the following:

  1. Handle "overflow": We can provide dates like October 32, and Golang will help us correct it (to November 1)—you can check the source code of AddDate, which simply operates on the relevant values.
  2. Handle leap years.
  3. Handle time zones.
  4. Use Unix timestamps to generate a time with hasMonotonic=0.

time.Parse()#

// To reduce the length of the article and highlight the key points, some code and comments have been edited.

func Parse(layout, value string) (Time, error) {
	return parse(layout, value, UTC, Local)
}

func parse(layout, value string, defaultLocation, local *Location) (Time, error) {
	...
	return Date(year, Month(month), day, hour, min, sec, nsec, defaultLocation), nil
}

This has been edited more thoroughly; its essence is that after parsing the format, it calls Date, so the generated time is also hasMonotonic=0.

Summary#

We can see that among the four most commonly used ways to construct time (Now, Date, Unix, Parse), only Now stores the information of the monotonic clock (hasMonotonic=1).

This storage has the most important advantage: we can use time.Now() - time.Now() to calculate the elapsed time between two executions without worrying about time travel.

Time Difference#

Subtracting Time#

In fact, all difficulties are resolved once we understand the structure of time.Time. After designing the structure, if you were to write Sub yourself, you would write it this way.

Without further ado, let's look at the code for Sub.

// Sub returns the duration t-u. If the result exceeds the maximum (or minimum)
// value that can be stored in a Duration, the maximum (or minimum) duration
// will be returned.
// To compute t-d for a duration d, use t.Add(-d).
func (t Time) Sub(u Time) Duration {
	if t.wall&u.wall&hasMonotonic != 0 {
		te := t.ext
		ue := u.ext
		d := Duration(te - ue)
		if d < 0 && te > ue {
			return maxDuration // t - u is positive out of range
		}
		if d > 0 && te < ue {
			return minDuration // t - u is negative out of range
		}
		return d
	}
	d := Duration(t.sec()-u.sec())*Second + Duration(t.nsec()-u.nsec())
	// Check for overflow or underflow.
	switch {
	case u.Add(d).Equal(t):
		return d // d is correct
	case t.Before(u):
		return minDuration // t - u is negative out of range
	default:
		return maxDuration // t - u is positive out of range
	}
}

The core logic is probably what you think: if both times are hasMonotonic=1, just calculate the difference in the monotonic clock values—subtract the two ext values; otherwise, calculate the difference in the calendar clocks.

Do you remember our previous extreme analysis? In extreme cases, regardless of which branch of hasMonotonic we reach, there will always be a possibility of overflow—two time intervals are too large, exceeding the maximum capacity of int64 nanoseconds, which is 292.47 years.

I recommend you carefully read the code above, especially to think about why the methods for detecting overflow differ in the two cases.

Comparison#

Zero Value — IsZero#

When we directly initialize a Time, it is a zero value.

var t time.Time

The initialization of Time has no magic; it is the same as the initialization of other structs in Go—their fields are all assigned 0. Therefore, according to the rules, it has hasMonotonic=0, so it uses ext to store seconds, and the third part of wall to store nanoseconds, both of which are also 0. Thus, the zero value of time is January 1, year 1, 00:00:00.000000000 UTC.

Since this time is rarely encountered in practice, the zero value is usually considered "uninitialized time." Go provides the IsZero method to detect it.

// To reduce the length of the article and highlight the key points, some comments have been added.

// IsZero reports whether t represents the zero time instant,
// January 1, year 1, 00:00:00 UTC.
func (t Time) IsZero() bool {
	return t.sec() == 0 && t.nsec() == 0
}

// sec returns the time's seconds since Jan 1 year 1.
func (t *Time) sec() int64 {
	if t.wall&hasMonotonic != 0 { // This branch will not be reached for zero value
		return wallToInternal + int64(t.wall<<1>>(nsecShift+1))
	}
	return t.ext
}

// nsec returns the time's nanoseconds.
// This simply returns the third part of wall (30 bits of nanosecond time offset).
func (t *Time) nsec() int32 {
	return int32(t.wall & nsecMask)
}

const nsecMask = 1<<30 - 1

Its implementation looks a bit complicated, but in fact, considering that the value cannot be 0 when hasMonotonic=1, as long as this Time is not forcibly corrupted by some pointer manipulation, its wall's first and third parts are 0 + ext is 0, making the overall value 0.

Equality — Equal#

First, it is important to clarify: if you directly compare structs, then struct equality means time equality, but struct inequality does not necessarily mean time inequality. Therefore, when comparing times in Go, we should avoid using == and instead use the Equal method.

The reasons for struct inequality while time equality include: there may be two representations (hasMonotonic=0/1) corresponding to the same value, and there may be two time zones corresponding to the same value.

The focus of this article is not on time zones; time zones do not affect the wall and ext in Time (both are stored using UTC values), only the loc field.

// Equal reports whether t and u represent the same time instant.
// Two times can be equal even if they are in different locations.
// For example, 6:00 +0200 and 4:00 UTC are Equal.
// See the documentation on the Time type for the pitfalls of using == with
// Time values; most code should use Equal instead.
func (t Time) Equal(u Time) bool {
	if t.wall&u.wall&hasMonotonic != 0 {
		return t.ext == u.ext
	}
	return t.sec() == u.sec() && t.nsec() == u.nsec()
}

We can see that the implementation of Equal in Go also follows the "monotonic clock priority principle."

Additional Information#

The core of this article—time.Time—has been explained (well, except for time zones). In this section, I want to write about things that may not be needed in usage but are worth knowing from a technical perspective.

ᕦ(•́~•̀ ) The additional content is actually more than the main text.

How is time.Now implemented?#

It has been mentioned that the implementation of time.Now is as follows, but have you noticed...

// Provided by package runtime.
func now() (sec int64, nsec int32, mono int64)

This now function is directly omitted 😂, leaving only a line implemented by runtime.

Yes, it is indeed implemented by runtime, with the real implementation in the runtime's time_now function, and furthermore, the implementations differ between Linux Amd64 and other systems.

//go:build !faketime && (windows || (linux && amd64))

//go:linkname time_now time.now
func time_now() (sec int64, nsec int32, mono int64)
//go:build !faketime && !windows && !(linux && amd64)

//go:linkname time_now time.now
func time_now() (sec int64, nsec int32, mono int64) {
	sec, nsec = walltime()
	return sec, nsec, nanotime()
}

On Linux Amd64, the entire time_now is implemented in assembly. The following code is a version with my comments added; you can read the original version here.

// To reduce the length of the article and highlight the key points, some code and comments have been edited.

// func time.now() (sec int64, nsec int32, mono int64)
// $16-24 indicates that the function requires 16 bytes of stack space and 24 bytes of return value space
TEXT time·now<ABIInternal>(SB),NOSPLIT,$16-24
	// Prepare vDSO
    ...

	// (If not g0) switch to g0
	...

noswitch: // Logic to get time (prefer using vDSO)
	SUBQ	$32, SP		// Space for two time results
	ANDQ	$~15, SP	// Align for C code

	MOVL	$0, DI // CLOCK_REALTIME (get calendar clock)
	LEAQ	16(SP), SI
	MOVQ	runtime·vdsoClockgettimeSym(SB), AX
	CMPQ	AX, $0
	JEQ	fallback // Jump if failed
	CALL	AX

	MOVL	$1, DI // CLOCK_MONOTONIC (get monotonic clock)
	LEAQ	0(SP), SI
	MOVQ	runtime·vdsoClockgettimeSym(SB), AX
	CALL	AX

ret: // Successfully obtained, results are in the stack
	MOVQ	16(SP), AX	// realtime sec
	MOVQ	24(SP), DI	// realtime nsec (moved to BX below)
	MOVQ	0(SP), CX	// monotonic sec
	IMULQ	$1000000000, CX
	MOVQ	8(SP), DX	// monotonic nsec

	// Restore context
    ...

	// Return results
	// set result registers; AX is already correct
	MOVQ	DI, BX
	ADDQ	DX, CX // Calculate monotonic nanoseconds
	RET

fallback: // If CLOCK_REALTIME fails, it means vdso failed, use syscall to get
	MOVQ	$SYS_clock_gettime, AX
	SYSCALL

	MOVL	$1, DI // CLOCK_MONOTONIC
	LEAQ	0(SP), SI
	MOVQ	$SYS_clock_gettime, AX
	SYSCALL

	JMP	ret

In simple terms, on Linux Amd64, Go tries to use vDSO to obtain time information (including both calendar and monotonic clock times). If an error occurs, it falls back to a system call. The main purpose of vDSO is to reduce the time of system calls; you can read more about it in the Linux manual. In Go, in fact, only system calls related to time use vDSO: vdso_linux_amd64.go.

On non-Linux Amd64 systems, walltime() and nanotime() are used to obtain the calendar clock and monotonic clock, respectively.

The following code takes Linux Arm64 as an example.

func walltime() (sec int64, nsec int32)

//go:nosplit
func nanotime() int64 {
	return nanotime1()
}

func nanotime1() int64

Seeing this without a body indicates that it must be implemented in assembly 😂. Understanding the above Linux Amd64 implementation, the Arm64 implementation is quite similar (both use vDSO + syscall fallback), except that walltime and nanotime are not obtained in one function. You can click here to read the relevant source code.

How to Obtain the System Monotonic Clock?#

The time package does not provide a means to obtain the system monotonic clock, but runtime's nanotime is actually the system monotonic clock. You can wrap the following content into a helper function for use.

import (
	_ "unsafe"
)

//go:noescape
//go:linkname Monotonic runtime.nanotime
func Monotonic() int64

Or... use my extime.Monotonic 👀.

Round — Rounding + Removing Monotonic Clock Information#

Round is used for rounding.

// Round returns the result of rounding t to the nearest multiple of d (since the zero time).
// The rounding behavior for halfway values is to round up.
// If d <= 0, Round returns t stripped of any monotonic clock reading but otherwise unchanged.
//
// Round operates on the time as an absolute duration since the
// zero time; it does not operate on the presentation form of the
// time. Thus, Round(Hour) may return a time with a non-zero
// minute, depending on the time's Location.
func (t Time) Round(d Duration) Time {
	t.stripMono()
	if d <= 0 {
		return t
	}
	_, r := div(t, d)
	if lessThanHalf(r, d) {
		return t.Add(-r)
	}
	return t.Add(d - r)
}

// stripMono strips the monotonic clock reading in t.
func (t *Time) stripMono() {
	if t.wall&hasMonotonic != 0 {
		t.ext = t.sec()
		t.wall &= nsecMask
	}
}

You can use Duration to provide precision (for example, t.Round(time.Second)).

Additionally, it has a "side effect," which is that it will remove the monotonic clock information we mentioned earlier (the first line of stripMono). Therefore, besides rounding, if you want to remove the monotonic clock content, you can use t.Round(0).

Leap Seconds#

If you don't know about leap seconds, please refer to Wikipedia. Leap seconds are something that has always existed, caused countless incidents, and is about to be canceled. Because our predecessors made mistakes, we may no longer pay much attention to it, but you should at least be aware of it.

Leap seconds somewhat exceed the scope of this article—because Go does not consider leap seconds. In Go, you will not get a time like "23:59:60," and because monotonic clocks have now been introduced, there will be no more incidents when calculating time differences.

Wait, again? Please refer to the incident at Cloudflare in 2017, when Go's time.Now had not yet used the monotonic clock.

References#

go/time.go at go1.20.4 · golang/go

https://github.com/golang/go/issues/12914

Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.