how to insert timestamp beyond the limitation of keep in tdengine

Question

I found that tdengine has a parameter will create database. this definition "The KEEP parameter refers to the number of days to save a modified data file. " from tdengine's website https://www.taosdata.com/en/documentation/taos-sql#management. I think this parameter is very useful that there is no need to delete the history data. so i create a database which only keep 10 days.

CREATE DATABASE IF NOT EXISTS db_keep KEEP 10 PRECISION 'ms' ;
create table test_keep(ts timestamp,desc nchar(20));

after create db and table , I tried to insert some data into the table. the follows are my insert sqls.

insert into test_keep values(now,'now');
insert into test_keep values('2021-08-31 10:28:53.521','yesterday');
insert into test_keep values('2021-09-02 10:28:53.521','tomorrow');
insert into test_keep values('2021-08-25 10:28:53.521','6 days before');
insert into test_keep values('2021-09-20 12:28:53.521','20 days later');
insert into test_keep values('2021-08-21 10:28:53.521','10 days before');
insert into test_keep values('2021-08-11 10:28:53.521','20 days before');

While the lass three sql had execute error "DB error: Timestamp data out of range"

taos> insert into test_keep values(now,'now'); Query OK, 1 of 1 row(s) in database (1.024000s)

taos> insert into test_keep values('2021-08-31 10:28:53.521','yesterday'); Query OK, 1 of 1 row(s) in database (0.006000s)

taos> insert into test_keep values('2021-09-02 10:28:53.521','tomorrow'); Query OK, 1 of 1 row(s) in database (0.004000s)

taos> insert into test_keep values('2021-08-25 10:28:53.521','6 days before'); Query OK, 1 of 1 row(s) in database (0.004000s)

taos> insert into test_keep values('2021-09-20 12:28:53.521','20 days later');

DB error: Timestamp data out of range (0.005000s) taos> insert into test_keep values('2021-08-21 10:28:53.521','10 days before');

DB error: Timestamp data out of range (0.004000s) taos> insert into test_keep values('2021-08-11 10:28:53.521','20 days before');

DB error: Timestamp data out of range (0.004000s) taos>

I thought this because of my keep is to small, so i made it larger.

ALTER DATABASE db_keep KEEP 365;

and the I tried to insert the failed sql again I found cannot insert data some days later from now .

taos> insert into test_keep values('2021-09-20 12:28:53.521','20 days later');

DB error: Timestamp data out of range (0.005000s) taos> insert into test_keep values('2021-08-21 10:28:53.521','10 days before'); Query OK, 1 of 1 row(s) in database (0.004000s)

taos> insert into test_keep values('2021-08-11 10:28:53.521','20 days before'); Query OK, 1 of 1 row(s) in database (0.004000s)

I want to ask how to use keep and the how dose it limit the data's timestamp?

Answer 1

For timestamp in a database, the two configuration options is most reasonable:

1. keep: the longest days to keep in the database refer to current timestamp(3650 days by default). When the "newest" timestamp in a persist file goes out of the range, the persist file will be deleted. Timestamp older than now - keep will be treated as Timestamp out of range.
2. days: the time range to store data in a file, 10 days by default. Newer timestamp would not be larger than now+ days.

So, the acceptale timestamp range in a database will be [now - keep, now + days].

Answer 2

For past data, the timestamp value cannot exceed (current_time - keep). Meanwhile, for future data, the timestamp value cannot exceed (current time + days).

Since my reputation is not enough for inserting images here, please refer to reference link below for details (grey means can NOT be inserted).

Reference:

1. https://segmentfault.com/a/1190000040617572