phacai.com

Some notes about iSCSI

Mr. One — Wed, 09 May 2012 16:22:47 +0000

I am new to storage technologies, but the plan is to build a customized SAN (storage area network). The intelligence will be how blocks are managed, such as automatic duplication or deduplication.

In order to test out the idea, but without any possession of fibre channel, I will just stick to existing Ethernet infrastructure. Therefore, the protocol of choice is iSCSI. Fortunately, this is now supported by Linux kernel.

For iSCSI, there are two existences: initiator and target. An initiator sends SCSI requests, which will be wrapped into TCP/IP requests. A target is responsible for serving requests and communicating with block devices. In other words, initiator is like a SCSI adapter that is installed on a client machine, while a target is a storage server instance. Apparently, target does all the heavy lifting.

The current implementation of iSCSI target on linux is called “LIO Target”, developed and open-sourced by RisingTide Systems LLC, featuring a German CEO/Co-founder. It is in Linux kernel since version 2.6.38. The current kernel version is 3.3.4. So it has been there for a while, which means it’s getting stable.

LIO Target does not only support iSCSI. It also includes support for Fibre Channel, Infiniband, etc. But I will only need its iSCSI module.

This post will be expanded with more information until I figure out which part of LIO Target should be modified to include new functionality.

1. Setup
(1) Target: According to this document, setup of a LIO target is handled by a dedicated tool called Targetcli.
(2) Initiator: The initiator from the same company, Rising Tide Systems, is called Core-iSCSI.

No related posts.

20K visits milestone for Phacai.com

Mr. One — Wed, 18 Apr 2012 15:48:34 +0000

In one and a half month, it will be a year since I started to use “ClustrMaps” for site statistics. And “phacai.com” has already served more than 20K (20,435 to be specific) visits! That’s around 80 visits per day.

Hope to double this number in the next year. Following is a screenshot I took a moment ago.

Best Multicore Posts of 2008 on MultiCoreInfo.com

Contractor rate for programmer (freelancer)

Mr. One — Fri, 13 Apr 2012 21:02:18 +0000

Just a note.

Here is a post that talks about the cost of building an iPhone app.

In this post, people (first and second top comments) mentioned some price rates for programming contractors: (1)$150/hr for the Twitterrific app; (2) $100-150/hr for the Barack Obama app. I don’t know if programmers could at a share of the app’s future revenue, which might be a non-trivial amount for a serious app.

A typical big company in Silicon Valley, pays his programmer at $50~100/hr ($8,000~16,000/month), but there will be other benefits such as medical insurance and annual bonus.

The choices on the two sides, freelancer or employee, sound both interesting.

No related posts.

Python code: merge multiple files into one multi-column file

Mr. One — Mon, 09 Apr 2012 19:09:45 +0000

Let’s begin with an example. Assume two input files, “cdf1″ and “cdf2″:

“cdf1″

0 0.0009765625
3 0.011474609375
18 0.013916015625
20 0.014404296875
21 0.091552734375
131 0.092041015625
239 0.162841796875
604 0.164306640625
605 0.166748046875
607 0.167724609375

“cdf2″

5 0.011474609375
10 0.013916015625
20 0.014404296875
21 0.091552734375
131 0.092041015625
239 0.162841796875
604 0.164306640625
605 0.166748046875
607 0.167724609375
946 0.168212890625

Both files contain two columns. I want to merge them into the following single file:

0 0.0009765625 5 0.011474609375
3 0.011474609375 10 0.013916015625
18 0.013916015625 20 0.014404296875
20 0.014404296875 21 0.091552734375
21 0.091552734375 131 0.092041015625
131 0.092041015625 239 0.162841796875
239 0.162841796875 604 0.164306640625
604 0.164306640625 605 0.166748046875
605 0.166748046875 607 0.167724609375
607 0.167724609375 946 0.168212890625

As we can see the output file contains 4 columns, coming from the two input files.

This can be accomplished by the following script. It can handle various number of input files, not just two (refer to USAGE).

Download: merger.py

#!/usr/bin/env python
import os, sys, string
if len(sys.argv)<=2:
print "USAGE: ./cdf-merger.py outfile cdf1 cdf2 …";
exit(0)
outfile=sys.argv[1]
fdout=open(outfile,"w")
if len(sys.argv)==3:
cdf1=sys.argv[2]
os.system("cp "+cdf1+" "+outfile)
exit(0)
n_cdfs=len(sys.argv)-2
fnames=[]
fds=[]
curstr=[]
for i in range(2, 2+n_cdfs):
fnames.append(sys.argv[i])
fds.append(open(sys.argv[i], "r"))
curstr.append("NULL")
for i in range(0, 4097): #the final file will contain no more than 4097 lines
count=0
for f in range(len(fds)): #read one line from each file
tmpline = fds[f].readline().rstrip("\n")
if len(tmpline)>0:
curstr[f]=tmpline
else:
count+=1
if count==n_cdfs:
break #all input files have reached their ends
for f in range(len(fds)):
if f>0:
fdout.write(" ")
fdout.write(curstr[f])
fdout.write("\n")
fdout.close()

Some performance numbers of PostgresSQL running TPC-H (DBT-3)

Mr. One — Fri, 02 Mar 2012 02:36:57 +0000

Now It’s hard to find some real performance numbers of PostgreSQL. The original link storing such information does not work anymore: http://developer.osdl.org/markw/postgrescvs/. Try it yourself, you will be redirected.

I found an old post on the Internet. But this guy used an ancient PostgreSQL, version 8.1.4, to run TPC-H (DBT-3 implementation). I don’t know how much performance will be improved if we use nowadays’ 9* versions. Anyway, the link is here: link

Slow queries in throughput test of TPC-H on PostgreSQL

Mr. One — Wed, 29 Feb 2012 03:16:46 +0000

Queries 9, 18, 21 are extremely slow. Query 18 is quite fast when I run it separately, but slowed down a lot in throughput test.

The relative performance numbers are very close to the case of HP DL980. (link2)

Nested loop when running TPC-H on PostgreSQL

Mr. One — Tue, 28 Feb 2012 01:08:39 +0000

I confirmed the observation by David Gruzman that for some queries in TPC-H, PostgreSQL is very slow because too much nested loop join is used. Using hash join instead will greatly improve performance.

Here is my configuration

Scale Factor: 10
Dataset size: 16GB (7 indexes)
DRAM size: 2GB
HDD: one Seagate Cheetha 15.7K rpm

The indexes were built as specified in one of my previous posts: link. My machine is equipped with 8GB of memory, but I used Linux boot option “mem” to limit the size available to system.

Among all the 22 queries of TPC-H, the following used queries used nest loop join

2 4 7 8 9 17 20 21

When disabling nest loop join, the performance of the following queries were improved. Q9 and Q21 were especially improved.

4 8 9 21

How to trim your SSD under linux?

Mr. One — Thu, 23 Feb 2012 01:30:22 +0000

We know in the latest file system EXT4, when a file is deleted, a trim command will be automatically issued so that the underlying device will know which blocks are no longer used. SSDs will benefit from TRIM because the garbage collection overhead can be reduced.

Sometimes you may want to trim your SSD manually, maybe because your are not running on EXT4. Here is how you can do this. The basic tool is “hdparm”. With this tool, we can trim an SSD at the granularity of blocks. First, check your tool by typing

hdparm --help

Towards the end of the list, you should expect the following two options

 --trim-sector-ranges
 --trim-sector-ranges-stdin

If you don’t see them, your tool is outdated. I have two machines, each running a different linux distribution. Fedora Core 13 has the right “hdparm” out of the box, but Ubuntu 10.4 does not.

Assume your hdparm is up-to-date. With the first option “–trim-sector-ranges”, you can trim your device like this:

hdparm --trim-sector-ranges 1000:4 7894:16 /dev/sdz

As you can image, this is simple. In the above command, it will trim two ranges, starting at Sector 1000 and 7894 respectively. In the first region, 4 sectors will be trimmed, and in the second region, 16 sectors will be trimmed. For this command, a sector means 512 bytes.

Although very simple, it will become messy if you want to trim many sectors at once. Sometimes, you may have to specify a very large continuous range as many consecutive ranges. In this case, you will need the second option. Following is a script that shows how to use this option.

t1mb=`expr 1024 \* 1024 / 512`
t1gb=`expr 1024 \* 1024 \* 1024 / 512`
sizes="$t1gb"
for sz in $sizes;do
echo "# of sectors: $sz;"
st=0
ts1=`date +%s`
while [ $st -lt $sz ];do
echo $st:$bstep
st=`expr $st + $bstep`
done | time hdparm –please-destroy-my-drive –trim-sector-ranges-stdin /dev/sdd
ts2=`date +%s`
dur=`expr $ts2 – $ts1`
echo "Total time: $dur seconds;"
done

Note: if your SSD is quite old, such as the Intel 25E, 25M series, they may not support TRIM, which is a relatively new term. If you send trim commands to such devices, you will see error messages such as “FAILED: Input/Output error”.

NAT on a Ubuntu machine

Mr. One — Tue, 14 Feb 2012 21:08:55 +0000

My ubuntu machine has two NICs. eth0 connects to the outside world, and eth1 connects to my internal network. To enable Internet access for internal network, follow these steps.

su
echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
/sbin/iptables -A FORWARD -i eth0 -o eth1 -m state –state RELATED,ESTABLISHED -j ACCEPT
/sbin/iptables -A FORWARD -i eth1 -o eth0 -j ACCEP
/sbin/iptables -A INPUT -p tcp -m tcp –sport 53 –dport 1024:65535 -m state –state ESTABLISHED -j ACCEPT
/sbin/iptables -A INPUT -p udp -m udp –sport 53 –dport 1024:65535 -m state –state ESTABLISHED -j ACCEPT
/sbin/iptables -A OUTPUT -p tcp -m tcp –sport 1024:65535 –dport 53 -m state –state NEW,ESTABLISHED -j ACCEPT
/sbin/iptables -A OUTPUT -p udp -m udp –sport 1024:65535 –dport 53 -m state –state NEW,ESTABLISHED -j ACCEPT

Configure the machines in the internal network, set their gateways to the (internal) IP address of the Ubuntu machine.

To make sure these settings can survive a reboot.

(1) Open /etc/sysctl.conf, assign “1″ to parameter “net.ipv4.ip_forward”;
(2) Open /etc/rc.local, add the rest of the command to this file, at the bottom but before “exit”.

No related posts.

Indexes created by dbt3 (TPC-H)

Mr. One — Tue, 07 Feb 2012 20:25:18 +0000

DBT3 creates many indexes: one primary index on the primary key of each table, and the following secondary indexes:

CREATE INDEX i_l_shipdate ON lineitem (l_shipdate);
CREATE INDEX i_l_suppkey_partkey ON lineitem (l_partkey, l_suppkey);
CREATE INDEX i_l_partkey ON lineitem (l_partkey);
CREATE INDEX i_l_suppkey ON lineitem (l_suppkey);
CREATE INDEX i_l_receiptdate ON lineitem (l_receiptdate);
CREATE INDEX i_l_orderkey ON lineitem (l_orderkey);
CREATE INDEX i_l_orderkey_quantity ON lineitem (l_orderkey, l_quantity);
CREATE INDEX i_c_nationkey ON customer (c_nationkey);
CREATE INDEX i_o_orderdate ON orders (o_orderdate);
CREATE INDEX i_o_custkey ON orders (o_custkey);
CREATE INDEX i_s_nationkey ON supplier (s_nationkey);
CREATE INDEX i_ps_partkey ON partsupp (ps_partkey);
CREATE INDEX i_ps_suppkey ON partsupp (ps_suppkey);
CREATE INDEX i_n_regionkey ON nation (n_regionkey);
CREATE INDEX i_l_commitdate ON lineitem (l_commitdate);

However, not every index will be used. Generating more than necessary indexes will increase the data set size. I analyzed in my settings, only 14 indexes are used (4 primary and 10 secondary):

i_l_orderkey
i_l_partkey
i_l_receiptdate
i_l_shipdate
i_l_suppkey_partkey
i_o_custkey
i_o_orderdate
i_ps_partkey
i_ps_suppkey
i_s_nationkey
pk_customer
pk_orders
pk_part
pk_supplier

To create a database of a minimum size, with only (most) necessary indexes created, please refer to a previous post of mine: Preparations to use TPCH data on Postgresql.