Journal

 

This is weekly record.

 

Week 1
I had a meeting with Sarah and Mark. This idea was come from Mark and he wanted the system can distinguish real-time request and non real-time request and the system can have different algorithm for real-time request. He wanted that we can make sure that a request can be issued before a certern time point. Sarah gave the idea that we can do this work in device mapper layer.
 
So the original plan was to do some modification in dm-linear to create an entry in /proc and can show all the bios, then do some modification in file system to add some control for flush based on kernel timer and transaction size.
 
I actually didn't understand well at that time because I have no idea how device mapper works. I read some books and do some search on line. I tried to get device mapper work however it didn't work because I didn't know that I should have a empty partition before using device mapper.

Week 2
I did the initial presentation. I had a meeting with Sarah and she helped me to figure out how to set up dmsetup. I already had an empty partition, /dev/sda10, however I still can not map it. It turned out to be that I haven't loaded the necessary module and I need to first map it, then format it at the first time. After this meeting, I finally figured out how to map a device and how to use dmsetup tool.
 
I then found a really good material online talking about how device mapper works. It is in Chinese which is easy for me to read. Actually some pictures I used in my final presentation all from there. From this material, I knew how dmsetup tool work with the kernel module.
 
However I still can not make a separate module from dm-linear. First I didn't mark device mapper as compiled to modules which makes me to update the kernel image every time I made a little change. Sceond, I could not have dmsetup tool working with my dm-statistic module. What I can do is only to print put an information in the system log to say that I see one bio comes in in dm-linear. When I did the same thing in dm-statistic, it didn't work.

Week 3
Since I can not figure out how to make it work, so I met with Sarah again. This time with Sarah's help, I made big progress. Sarah suggested me to do a little in Kconfig to add one more item and then add one line in Makefile to make dm-statistic a independent module because that's what she did for her true-erase project. I followed her steps and it worked. Now the dmsetup can fully work with my new modules.
 
I created a /proc file called statistic and fill the buf with some information about the bio just came in. It worked and I can see those content in the /proc/statistic file, however I didn't consider a important thing. It will create a new /proc/statistic file every time it maps a bio. So I had a lot of /proc/statistic files in procfs and each contains the information about one bio which is not we wanted. What we want is to have only one file and it contains all the bio information.
 
I used a really dumped approach to slove this problem. After I create an entry, I filled the buffer, then I remove the entry. This time because it was so fast that I can not see any /proc/statistic file. I solved this problem in next week.

Week 4
This week is the most important week for me and for this project. I met Sarah again. This time I download a newest kernel version which is linux 3.8.7 and start do all the thing in this kernel. I first removed all the unnecessary modules and compiled the kernel. So the kernel is really fast, however the cost is I even didn't have the wireless network module. It compiles and load/reload modules really fast. I put the .config file in the dowload page.
 
I made too major progress in the project. First I solved the multiple /proc/statistic files and non /proc/statistic files problem. Actually it was really easy. You can create the file only once, outside the map function, and it map function, just do fill. And I remove the entry when the exit() function was called. However when I test it, there was another problem. When I had too many bios, it can not be read properly in /proc/statistic. After I searched the internet, I found out that it was because procfs can only handle no more than one page size. Normal the page size is 4MB, so if the content you want to store in procfs file is more than 4MB, the read operation can not work correctly.
 
The second major progress is that I sloved the page size limitation problem above. In this kind of situation, the right way to do it using seq_file interface. I searched online and there are very good tutorals and examples. I followed those materials and solved the problem in some level.

Week 5
Actually I came up with a different idea to do controling. Instead of doing this work in file system, can we do it in some other ways. The first idea is to use /proc/sys so the kernel module can get some configuration from some file and those values actually controls the behavior of the device mapper layer. However Sarach still suggeted me to do the work in file system. She gave me a very good guide so I wan't be so confuesed when tracing the data flow from file system to jbd2. I tried but I failed to do it.
 
Then I thought about other alternatives and I really got a very good idea. Since we are writing a new module in device mapper, why can't we do the controling work in the same module. Then I try to add this function just in my dm-statistic module. luckily it worked and i think it is easier than do the controling work in any other place. I showed this approach to Sarah and she also thinks this is a good try because it simplify all the things a lot. Even though I bypass the most difficult part, however I still achieve the goal.
 
I also improved my seq_file part to avoid a memory leak. Because in previous implementation I forgot to add clean function for the list structure. I also re-formatiing the bio information in /proc/statistic file.
 
Another work is I wrote a user program to get the count of bio from /proc/statistic, so that user can use this program to check the amount of each type of bio in a peraiod of time. I provided to checking ways, one is to check during last x second and the other is to check from x second to y second.

Week 6
This is the demo week and I did the demo on Tuesday. Sarah suggested me to add some extra functionalities like reset time, export functions to the global kernel and more descriptive information about bio. Those are not very difficult work. Only one thing is when I do reset time, it also cleaned all the bio structure that already in the list. Because it makes no sence that some bio even exsist before start time.