Project files list generate

Requirment

最近想了解一个纯正的面向对象的语言的设计思路，并顺便掌握一个方便的脚本语言以供日常使用，这样Ruby就进入了我的视线。看了2天的Ruby User guide,感觉应该练练手，否则可没有什么加深印象的过程。想起以前做的在Linux项目下生成一个file list，只包含哪些参与编译的c文件和h文件，这样我们用IDE来管理工程文件就没有必要把无关的程序加进来，方便我们查找和观看代码。 原来做的在编译时加入符号参数，最后对elf文件进行解析，最后生成文件列表。但是，这样子会有一些文件没有包含进来，原因不明。
但是我发现编译完后在各个源文件的目录下都会有相应的.*cmd文件，如在目录"driver/mxc/ipu3/"下面,一个文件是.ipu_disp.o.cmd,打开文件,内容如下:

cmd_drivers/mxc/ipu3/ipu_disp.o := /media/disk1/Android/androidr10.2/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-gcc -Wp,-MD,drivers/mxc/ipu3/.ipu_disp.o.d  -nostdinc -isystem /media/disk1/Android/androidr10.2/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/../lib/gcc/arm-eabi/4.4.3/include -I/media/disk1/Android/huaweiPCBA/kernel/arch/arm/include -Iinclude  -include include/generated/autoconf.h -D__KERNEL__ -mlittle-endian -Iarch/arm/mach-mx5/include -Iarch/arm/plat-mxc/include -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -marm -mabi=aapcs-linux -mno-thumb-interwork -funwind-tables -D__LINUX_ARM_ARCH__=7 -march=armv7-a -msoft-float -Uarm -Wframe-larger-than=1024 -fno-stack-protector -fomit-frame-pointer -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack   -D"KBUILD_STR(s)=\#s" -D"KBUILD_BASENAME=KBUILD_STR(ipu_disp)"  -D"KBUILD_MODNAME=KBUILD_STR(mxc_ipu)"  -c -o drivers/mxc/ipu3/.tmp_ipu_disp.o drivers/mxc/ipu3/ipu_disp.c

deps_drivers/mxc/ipu3/ipu_disp.o := \
  drivers/mxc/ipu3/ipu_disp.c \
  include/linux/types.h \
    $(wildcard include/config/uid16.h) \
  /media/disk1/Android/customer/kernel/arch/arm/include/asm/types.h \
  include/asm-generic/bitsperlong.h \
/media/disk1/Android/androidr10.2/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/../lib/gcc/arm-eabi/4.4.3/include/stdarg.h \

很明显,这个是依赖文件列表,我们可以看到gcc -MD这个是产生依赖文件的参数.所以我们只要处理这个文件即可以把参与编译的文件及其头文件产生出我们需要的列表出来.

Install ruby

apt-get install ruby1.9.1

It will install ruby in /usr/bin directory

Build Linux Kernel

Enter kernel directory, we need build it to generate dependency file.

$ make distclean
$ make imx5_defconfig
$ make uImage -j2

Analysis dependency file

我们的目的要把不必要的信息去掉，调整格式，使其对齐。我们来看上面那个.ipu_disp.o.cmd文件，每个依赖关系都是单独一行，这样就很好处理，Ruby的文件操作可以按行读取文件,从Ruby的文档里面可以找到如下语句.

lines = File.readlines(file_name)
lines.each { |line| block}

第一行代码是按行读取file_name的文件内容,存放在lines里面;第二行是对lines里面的每一行进行处理,就是那个"each",大括号"{}"里面放置我们要执行的代码.
我们看文件中包含 "$(wildcard"以及":="是没有必要要的. Ruby处理这些依然很方便:

lines.each { |line|
    if (line.include?("$") == true)
    elsif(line.include?(":=") == true) 
    else
    end
}

上面的代码通过.include?()来剔除包含"$"和":="的行,ruby中任何东西都是对象.
文件里面有这样行:

  /media/disk1/Android/customer/kernel/arch/arm/include/asm/types.h \
/media/disk1/Android/androidr10.2/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/../lib/gcc/arm-eabi/4.4.3/include/stdarg.h \

第一行是包含绝对路径的头文件,我们想像其他行那样是相对路径.第二行是标准库文件头,这个我们不需要包含进来,可以去掉.
"/media/disk1/Android/customer/"是我们当前目录,我们可以通过比较当前目录来判断,用空字符来替换它.

$working_directory = Dir::pwd    # Get current work directory
if line.include?($working_directory) then
    line = line.gsub($working_directory, "") # replace the working directory with null 
end

每行结尾有个转接符号"\",还有些行左边有空格,要去掉.第二行标准库文件头,由于不想和具体目录相关,就直接判断第一个字符是否有"/"

line = line.gsub("\\","");    # Remove last "\"
line = line.lstrip            # Remove left space characters
if(line[0] != "/") then       # ignore those lines which first character is "/"
...
end

Management file lists

我们需要存储那些文件列表,同时又要把重复的行去掉,我们需要用Hash来处理,Ruby同样有相应的功能, Hash.new.

$file_counter = 0
$database = Hash.new
def add_item(line)
    if ($database.has_key?(line) == true) then
  return
    end
    $database[line] = $file_counter
    $file_counter = $file_counter + 1
end 

Browse whole project directory to find out all the dependency files.

We use Dir.glob("**/.*cmd") to find out all the dependency files recursive.

file_list = Dir.glob("**/.*cmd")  #  Find all cmd files
file_list.each{|file|             #  Handle each file 
    puts(file)
    handle_cmd_file(file)
}

Save the project files list

Use File.new to create output file, with Hash table, use outfile.write key into output file

    outfile = File.new("prj-list.txt","w")
    $database.each {|key, value| 
  outfile.write key
    }
    outfile.close

Whole ruby code

After testing, it can generate project files list which can be used by IDE, about 2000 files which involved into build.

# Ruby study
$working_directory = Dir::pwd
$file_counter = 0
$database = Hash.new
def add_item(line)
    if ($database.has_key?(line) == true) then
        return
    end
    $database[line] = $file_counter
    $file_counter = $file_counter + 1
end 

def save_items
    outfile = File.new("prj-list.txt","w")
    $database.each {|key, value| 
 outfile.write key
    }
    outfile.close
end

def handle_cmd_file(file_name)
    lines = File.readlines(file_name)
    lines.each { |line|
  if (line.include?("$") == true)
 elsif(line.include?(":=") == true) 
        else
     if line.include?($working_directory) then
   line = line.gsub($working_directory, "")
            end
     line = line.gsub("\\",""); 
     line = line.lstrip 
     if(line[0] != "/") then 
  add_item(line)
     end
 end
    }
end


def check_cmd_files
    file_list = Dir.glob("**/.*cmd")
    file_list.each{|file|
 puts(file)
 handle_cmd_file(file)
    }
end

puts($working_directory)
check_cmd_files
save_items
puts($file_counter)