w*s
发帖数: 7227 | 1 【 以下文字转载自 Linux 讨论区 】
发信人: wds (净洗前尘，从头再来), 信区: Linux
标 题: system design question
发信站: BBS 未名空间站 (Thu Oct 18 16:11:17 2012, 美东)
in my linux embedded board, i have 10000 person's records in the database.
in the web front end, you can view the 1st 100 person's records.
Now if you click "next page", it'll display record from 101 to 200.
To get the records from database, you pull all data into the memory and
return 100 records to caller. Certainly this uses plenty of memory.
Now if you and your wife are both browsing this webpage, worst case i need
the memory for twice of the whole database.
If you have 10 wives accessing this page, my code will be in trouble, as it
could need 10 times of memory.
Now
1. do i have to do cache in the backend (forget about web front end) for
database info ? can i just return "busy" if 10 ppl are opening database at
the same time ?
2. what's the common way to handle this ? | b***i
发帖数: 3043 | 2 我认为你可以根据配置，需求来优化。你的内存是几十k，还是几G?
比如，是否可以index?
还有，mem cache是否是singleton?一份就够了，对读来说。
【在 w*s 的大作中提到】
: 【 以下文字转载自 Linux 讨论区 】
: 发信人: wds (净洗前尘，从头再来), 信区: Linux
: 标 题: system design question
: 发信站: BBS 未名空间站 (Thu Oct 18 16:11:17 2012, 美东)
: in my linux embedded board, i have 10000 person's records in the database.
: in the web front end, you can view the 1st 100 person's records.
: Now if you click "next page", it'll display record from 101 to 200.
: To get the records from database, you pull all data into the memory and
: return 100 records to caller. Certainly this uses plenty of memory.
: Now if you and your wife are both browsing this webpage, worst case i need
| d****n
发帖数: 1637 | 3 create a proxy before user can access the real database.(assuming all
operations are read only.)
in this proxy , use a hash or Tree data structure, so that, each time a user
asking his data, check the hash entries first.
if the 100 queries are not in the hash or partially not in the hash, start
getting them from the database, otherwise, if all the queries already in the
hash, bring them to the the front end.
the proxy need to be a high performance capable designed binary code and
keep itself in the system as a daemon thread.
you can add garbage cleaning or data expired control mechanism into it too.
And more...
【在 w*s 的大作中提到】
: 【 以下文字转载自 Linux 讨论区 】
: 发信人: wds (净洗前尘，从头再来), 信区: Linux
: 标 题: system design question
: 发信站: BBS 未名空间站 (Thu Oct 18 16:11:17 2012, 美东)
: in my linux embedded board, i have 10000 person's records in the database.
: in the web front end, you can view the 1st 100 person's records.
: Now if you click "next page", it'll display record from 101 to 200.
: To get the records from database, you pull all data into the memory and
: return 100 records to caller. Certainly this uses plenty of memory.
: Now if you and your wife are both browsing this webpage, worst case i need
| g*****g
发帖数: 34805 | 4 Did I miss something? Why can't you do pagination in your DB query like
you did in your UI?
【在 w*s 的大作中提到】
: 【 以下文字转载自 Linux 讨论区 】
: 发信人: wds (净洗前尘，从头再来), 信区: Linux
: 标 题: system design question
: 发信站: BBS 未名空间站 (Thu Oct 18 16:11:17 2012, 美东)
: in my linux embedded board, i have 10000 person's records in the database.
: in the web front end, you can view the 1st 100 person's records.
: Now if you click "next page", it'll display record from 101 to 200.
: To get the records from database, you pull all data into the memory and
: return 100 records to caller. Certainly this uses plenty of memory.
: Now if you and your wife are both browsing this webpage, worst case i need
| b*******s
发帖数: 5216 | 5 cache到用户端去
【在 w*s 的大作中提到】
: 【 以下文字转载自 Linux 讨论区 】
: 发信人: wds (净洗前尘，从头再来), 信区: Linux
: 标 题: system design question
: 发信站: BBS 未名空间站 (Thu Oct 18 16:11:17 2012, 美东)
: in my linux embedded board, i have 10000 person's records in the database.
: in the web front end, you can view the 1st 100 person's records.
: Now if you click "next page", it'll display record from 101 to 200.
: To get the records from database, you pull all data into the memory and
: return 100 records to caller. Certainly this uses plenty of memory.
: Now if you and your wife are both browsing this webpage, worst case i need
| w*s
发帖数: 7227 | 6 this is a file based database, i cannot do query based on range ...
if i understand ur q ...
【在 g*****g 的大作中提到】
: Did I miss something? Why can't you do pagination in your DB query like
: you did in your UI?
| g*****g
发帖数: 34805 | 7 I don't see why not as long as you have random access to the file. If you
just need some pre-known queries. Build index yourself. If not, you should
look at general purpose DB. Embedded devices usually have very few
concurrent users and the query requirement is not dynamic.
【在 w*s 的大作中提到】
: this is a file based database, i cannot do query based on range ...
: if i understand ur q ...
|
|
