GitHub

Retime

The retime function allows you to retime, i.e. change the timestamps of a TimeArray, similar to what Matlab's retime does.

using Plots, Dates, TimeSeries
gr()
timestamps = range(DateTime(2020, 1, 1), length = 7*24, step = Hour(1))
ta = TimeArray(timestamps, cumsum(randn(7*24)), [:a])
168×1 TimeArray{Float64, 1, DateTime, Vector{Float64}} 2020-01-01T00:00:00 to 2020-01-07T23:00:00
┌─────────────────────┬───────────┐
│                     │ a         │
├─────────────────────┼───────────┤
│ 2020-01-01T00:00:00 │  -1.13199 │
│ 2020-01-01T01:00:00 │  -1.13862 │
│ 2020-01-01T02:00:00 │ -0.336767 │
│ 2020-01-01T03:00:00 │  0.684708 │
│ 2020-01-01T04:00:00 │   0.92434 │
│ 2020-01-01T05:00:00 │   1.25309 │
│ 2020-01-01T06:00:00 │   2.08147 │
│ 2020-01-01T07:00:00 │   1.42497 │
│          ⋮          │     ⋮     │
│ 2020-01-07T17:00:00 │   16.4962 │
│ 2020-01-07T18:00:00 │   15.1241 │
│ 2020-01-07T19:00:00 │   15.6929 │
│ 2020-01-07T20:00:00 │   17.0689 │
│ 2020-01-07T21:00:00 │   15.6273 │
│ 2020-01-07T22:00:00 │    15.487 │
│ 2020-01-07T23:00:00 │   14.8381 │
└─────────────────────┴───────────┘
                   153 rows omitted

Using a new time step

retime(ta, Minute(15))
669×1 TimeArray{Float64, 2, DateTime, Matrix{Float64}} 2020-01-01T00:00:00 to 2020-01-07T23:00:00
┌─────────────────────┬──────────┐
│                     │ a        │
├─────────────────────┼──────────┤
│ 2020-01-01T00:00:00 │ -1.13199 │
│ 2020-01-01T00:15:00 │ -1.13199 │
│ 2020-01-01T00:30:00 │ -1.13199 │
│ 2020-01-01T00:45:00 │ -1.13199 │
│ 2020-01-01T01:00:00 │ -1.13862 │
│ 2020-01-01T01:15:00 │ -1.13862 │
│ 2020-01-01T01:30:00 │ -1.13862 │
│ 2020-01-01T01:45:00 │ -1.13862 │
│          ⋮          │    ⋮     │
│ 2020-01-07T21:30:00 │  15.6273 │
│ 2020-01-07T21:45:00 │  15.6273 │
│ 2020-01-07T22:00:00 │   15.487 │
│ 2020-01-07T22:15:00 │   15.487 │
│ 2020-01-07T22:30:00 │   15.487 │
│ 2020-01-07T22:45:00 │   15.487 │
│ 2020-01-07T23:00:00 │  14.8381 │
└─────────────────────┴──────────┘
                  654 rows omitted

Using new timestep vector

new_timestamps = range(DateTime(2020, 1, 1), DateTime(2020, 1, 2), step = Minute(15))
retime(ta, new_timestamps)
97×1 TimeArray{Float64, 2, DateTime, Matrix{Float64}} 2020-01-01T00:00:00 to 2020-01-02T00:00:00
┌─────────────────────┬──────────┐
│                     │ a        │
├─────────────────────┼──────────┤
│ 2020-01-01T00:00:00 │ -1.13199 │
│ 2020-01-01T00:15:00 │ -1.13199 │
│ 2020-01-01T00:30:00 │ -1.13199 │
│ 2020-01-01T00:45:00 │ -1.13199 │
│ 2020-01-01T01:00:00 │ -1.13862 │
│ 2020-01-01T01:15:00 │ -1.13862 │
│ 2020-01-01T01:30:00 │ -1.13862 │
│ 2020-01-01T01:45:00 │ -1.13862 │
│          ⋮          │    ⋮     │
│ 2020-01-01T22:30:00 │  5.93841 │
│ 2020-01-01T22:45:00 │  5.93841 │
│ 2020-01-01T23:00:00 │  6.04276 │
│ 2020-01-01T23:15:00 │  6.04276 │
│ 2020-01-01T23:30:00 │  6.04276 │
│ 2020-01-01T23:45:00 │  6.04276 │
│ 2020-01-02T00:00:00 │  7.63166 │
└─────────────────────┴──────────┘
                   82 rows omitted

Irregular timestamps

You can perform retime on irregularly spaced timestamps, both using a TimeArray with irregular timestamps or using a vector of irregular timestamps. Depending on the timestamps upsampling or downsampling is used. 

new_timestamps = vcat(
    range(DateTime(2020, 1, 1), DateTime(2020, 1, 2)-Minute(15), step = Minute(15)),
    range(DateTime(2020, 1, 2), DateTime(2020, 1, 3), step = Hour(1)),
)
retime(ta, new_timestamps)
121×1 TimeArray{Float64, 2, DateTime, Matrix{Float64}} 2020-01-01T00:00:00 to 2020-01-03T00:00:00
┌─────────────────────┬──────────┐
│                     │ a        │
├─────────────────────┼──────────┤
│ 2020-01-01T00:00:00 │ -1.13199 │
│ 2020-01-01T00:15:00 │ -1.13199 │
│ 2020-01-01T00:30:00 │ -1.13199 │
│ 2020-01-01T00:45:00 │ -1.13199 │
│ 2020-01-01T01:00:00 │ -1.13862 │
│ 2020-01-01T01:15:00 │ -1.13862 │
│ 2020-01-01T01:30:00 │ -1.13862 │
│ 2020-01-01T01:45:00 │ -1.13862 │
│          ⋮          │    ⋮     │
│ 2020-01-02T18:00:00 │   3.7975 │
│ 2020-01-02T19:00:00 │  2.26122 │
│ 2020-01-02T20:00:00 │  3.12281 │
│ 2020-01-02T21:00:00 │  4.42699 │
│ 2020-01-02T22:00:00 │  4.11843 │
│ 2020-01-02T23:00:00 │  3.36703 │
│ 2020-01-03T00:00:00 │  1.34923 │
└─────────────────────┴──────────┘
                  106 rows omitted

Upsampling

Interpolation is done using the upsample argument. If no data is directly hit, the specified upsample method is used. Available upsample methods are:

  • Linear() or :linear
  • Nearest() or :nearest
  • Previous() or :previous
  • Next() or :next
ta_ = retime(ta, Minute(15), upsample=Linear())
669×1 TimeArray{Float64, 2, DateTime, Matrix{Float64}} 2020-01-01T00:00:00 to 2020-01-07T23:00:00
┌─────────────────────┬───────────┐
│                     │ a         │
├─────────────────────┼───────────┤
│ 2020-01-01T00:00:00 │  -1.13199 │
│ 2020-01-01T00:15:00 │  -1.13364 │
│ 2020-01-01T00:30:00 │   -1.1353 │
│ 2020-01-01T00:45:00 │  -1.13696 │
│ 2020-01-01T01:00:00 │  -1.13862 │
│ 2020-01-01T01:15:00 │ -0.938154 │
│ 2020-01-01T01:30:00 │ -0.737692 │
│ 2020-01-01T01:45:00 │ -0.537229 │
│          ⋮          │     ⋮     │
│ 2020-01-07T21:30:00 │   15.5572 │
│ 2020-01-07T21:45:00 │   15.5221 │
│ 2020-01-07T22:00:00 │    15.487 │
│ 2020-01-07T22:15:00 │   15.3248 │
│ 2020-01-07T22:30:00 │   15.1626 │
│ 2020-01-07T22:45:00 │   15.0003 │
│ 2020-01-07T23:00:00 │   14.8381 │
└─────────────────────┴───────────┘
                   654 rows omitted
plot(ta)
plot!(ta_)

Downsampling

Downsampling or aggregation is done using the downsample argument. This applies a function to each interval not including the right-edge of the interval. If no data is present in the interval the specified upsample method is used. Available downsample methods are:

  • Mean() or :mean
  • Min() or :min
  • Max() or :max
  • Count() or :count
  • Sum() or :sum
  • Median() or :median
  • First() or :first
  • Last() or :last
ta_ = retime(ta, Hour(6), downsample=Mean())
28×1 TimeArray{Float64, 2, DateTime, Matrix{Float64}} 2020-01-01T00:00:00 to 2020-01-07T18:00:00
┌─────────────────────┬───────────┐
│                     │ a         │
├─────────────────────┼───────────┤
│ 2020-01-01T00:00:00 │ 0.0424612 │
│ 2020-01-01T06:00:00 │   1.14742 │
│ 2020-01-01T12:00:00 │   2.63933 │
│ 2020-01-01T18:00:00 │   4.17563 │
│ 2020-01-02T00:00:00 │   8.97395 │
│ 2020-01-02T06:00:00 │   8.25311 │
│ 2020-01-02T12:00:00 │   3.49143 │
│ 2020-01-02T18:00:00 │   3.51566 │
│          ⋮          │     ⋮     │
│ 2020-01-06T06:00:00 │   9.83863 │
│ 2020-01-06T12:00:00 │   14.4116 │
│ 2020-01-06T18:00:00 │   18.5118 │
│ 2020-01-07T00:00:00 │   18.8624 │
│ 2020-01-07T06:00:00 │   18.6206 │
│ 2020-01-07T12:00:00 │   17.3545 │
│ 2020-01-07T18:00:00 │   15.6397 │
└─────────────────────┴───────────┘
                    13 rows omitted
plot(ta)
plot!(ta_)

Extrapolation

Extrapolation at the beginning and end of the time series is done using the extrapolate argument. Available extrapolate methods are:

  • FillConstant(value) or :fillconstant
  • NearestExtrapolate() or :nearest
  • MissingExtrapolate() or :missing
  • NaNExtrapolate() or :nan
new_timestamps = range(DateTime(2019, 12, 31), DateTime(2020, 1, 2), step = Minute(15))
ta_ = retime(ta, new_timestamps, extrapolate=MissingExtrapolate())
193×1 TimeArray{Union{Missing, Float64}, 2, DateTime, Matrix{Union{Missing, Float64}}} 2019-12-31T00:00:00 to 2020-01-02T00:00:00
┌─────────────────────┬─────────┐
│                     │ a       │
├─────────────────────┼─────────┤
│ 2019-12-31T00:00:00 │ missing │
│ 2019-12-31T00:15:00 │ missing │
│ 2019-12-31T00:30:00 │ missing │
│ 2019-12-31T00:45:00 │ missing │
│ 2019-12-31T01:00:00 │ missing │
│ 2019-12-31T01:15:00 │ missing │
│ 2019-12-31T01:30:00 │ missing │
│ 2019-12-31T01:45:00 │ missing │
│          ⋮          │    ⋮    │
│ 2020-01-01T22:30:00 │ 5.93841 │
│ 2020-01-01T22:45:00 │ 5.93841 │
│ 2020-01-01T23:00:00 │ 6.04276 │
│ 2020-01-01T23:15:00 │ 6.04276 │
│ 2020-01-01T23:30:00 │ 6.04276 │
│ 2020-01-01T23:45:00 │ 6.04276 │
│ 2020-01-02T00:00:00 │ 7.63166 │
└─────────────────────┴─────────┘
                 178 rows omitted

Interpolation Methods

Available interpolation methods: Linear, Previous, Next, Nearest.

Aggregation Methods

Available aggregation methods: Mean, Min, Max, Count, Sum, Median, First, Last.

Extrapolation Methods

Available extrapolation methods: FillConstant, NearestExtrapolate, MissingExtrapolate, NaNExtrapolate.