[ldv-project] Queries in running the LDV.
Puneet Gupta
PUNEETG at xilinx.com
Fri Jul 5 10:41:09 MSK 2019
Hi Ilja,
I will not be able to apply patches , but I used a machine with lower kernel version.
* With kernel version 4.10 kernel, while installing the klever itself , klever-controller.service is not getting started properly due to which only klever manager page is getting shown ,
but when I run the tests nothing happens.
* I tried starting the klever-controller.service manually as well , but status is always active(exited)
2019-07-05 13:04:08 (utils.py:046) INFO> Execute command "service nginx start"
2019-07-05 13:04:08 (utils.py:046) INFO> Execute command "service klever-bridge start"
2019-07-05 13:04:10 (configure_controller_and_schedulers.py:064) INFO> (Re)configure production Klever Controller and Klever schedulers
2019-07-05 13:04:10 (utils.py:367) INFO> Stop services
2019-07-05 13:04:10 (utils.py:046) INFO> Execute command "service klever-controller stop"
2019-07-05 13:04:10 (utils.py:046) INFO> Execute command "service klever-native-scheduler stop"
2019-07-05 13:04:11 (configure_controller_and_schedulers.py:074) INFO> Configure Klever Controller
2019-07-05 13:04:11 (configure_controller_and_schedulers.py:090) INFO> Configure Klever Native Scheduler
2019-07-05 13:04:11 (configure_controller_and_schedulers.py:116) INFO> Configure Klever Native Scheduler Job Worker
2019-07-05 13:04:11 (configure_controller_and_schedulers.py:044) INFO> Configure Klever Native Scheduler Task Worker
2019-07-05 13:04:11 (utils.py:361) INFO> Start services
2019-07-05 13:04:11 (utils.py:046) INFO> Execute command "service klever-controller start"
2019-07-05 13:04:12 (utils.py:046) INFO> Execute command "service klever-native-scheduler start"
2019-07-05 13:04:14 (__init__.py:077) INFO> Finish execution of action "install" for Klever "production"
root at ipds-Precision-T7610:klever#
root at ipds-Precision-T7610:klever#
root at ipds-Precision-T7610:klever# service klever-controller status
● klever-controller.service - LSB: Klever computational resources and services controller
Loaded: loaded (/etc/init.d/klever-controller; generated)
Active: active (exited) since Fri 2019-07-05 13:04:12 IST; 1min 1s ago
Docs: man:systemd-sysv-generator(8)
Process: 5245 ExecStart=/etc/init.d/klever-controller start (code=exited, status=0/SUCCESS)
Jul 05 13:04:11 ipds-Precision-T7610 systemd[1]: Starting LSB: Klever computational resources and services controller...
Jul 05 13:04:11 ipds-Precision-T7610 klever-controller[5245]: * Starting Klever Controller
Jul 05 13:04:12 ipds-Precision-T7610 klever-controller[5245]: * Klever Controller failed to start
Jul 05 13:04:12 ipds-Precision-T7610 klever-controller[5245]: ...fail!
Jul 05 13:04:12 ipds-Precision-T7610 systemd[1]: Started LSB: Klever computational resources and services controller.
root at ipds-Precision-T7610:klever#’
Thanks,
Puneet
From: Ilja Zakharov <ilja.zakharov at ispras.ru>
Sent: Wednesday, July 3, 2019 2:39 PM
To: Puneet Gupta <PUNEETG at xilinx.com>
Cc: Alexey Khoroshilov <khoroshilov at ispras.ru>; ldv-project at linuxtesting.org
Subject: Re: [ldv-project] Queries in running the LDV.
EXTERNAL EMAIL
Hi Puneet,
Which version of the Linux kernel did you use at compilation? Seems that it requires a couple of patches to avoid problems with new kernels of the CIF tool which is used in the Klever framework. Please, find patches attached.
Just apply patches to the Linux kernel, clean and recompile your driver with clade-intercept and clade-all commands to create a new build base as you did it before, then install the build base with the update command and rerun your verification job in the web-interface.
Best,
Ilja Zakharov
вт, 2 июл. 2019 г. в 20:16, Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>:
Hi Ilja,
I tried both the options,
Now I am getting different error now.
In file included from ./include/linux/ipc.h:7:0,
from ./include/uapi/linux/sem.h:5,
from ./include/linux/sem.h:9,
from ./include/linux/sched.h:15,
from ./include/linux/cred.h:21,
from ./include/linux/seq_file.h:12,
from ./include/linux/pinctrl/consumer.h:17,
from ./include/linux/pinctrl/devinfo.h:21,
from ./include/linux/device.h:24,
from ./include/linux/dma-mapping.h:7,
from /scratch/puneet/klever/inst_dir/klever/build bases/clade/Storage/scratch/puneet/klever/src/libqdma/xdev.h:28,
from /scratch/puneet/klever/inst_dir/klever/build bases/clade/Storage/scratch/puneet/klever/src/libqdma/qdma_context.h:27,
from /scratch/puneet/klever/inst_dir/klever/build bases/clade/Storage/scratch/puneet/klever/src/libqdma/qdma_device.c:25:
./include/linux/rhashtable.h: In function ‘__rhashtable_lookup’:
./include/linux/rhashtable.h:646:1: error: LDV: ../../aspectator/gcc/c-family/ldv-convert.c: 4441: expression recursion limit is reached
Raise exception:
Traceback (most recent call last):
File "/scratch/puneet/klever/inst_dir/klever/core/core/components.py", line 427, in run
self.main()
File "/scratch/puneet/klever/inst_dir/klever/core/core/components.py", line 332, in callbacks_caller
ret = attr(*args, **kwargs)
File "/scratch/puneet/klever/inst_dir/klever/core/core/vtg/weaver/__init__.py", line 127, in weave
filter_func=core.vtg.utils.CIFErrorFilter())
File "/scratch/puneet/klever/inst_dir/klever/core/core/utils.py", line 185, in execute
raise CommandError('"{0}" failed'.format(cmd))
core.utils.CommandError: "cif" failed
I did do export PATH: <CIF_dir>:$PATH.
Thanks,
Puneet
From: Ilja Zakharov <ilja.zakharov at ispras.ru<mailto:ilja.zakharov at ispras.ru>>
Sent: Tuesday, July 2, 2019 9:03 PM
To: Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>
Cc: Alexey Khoroshilov <khoroshilov at ispras.ru<mailto:khoroshilov at ispras.ru>>; ldv-project at linuxtesting.org<mailto:ldv-project at linuxtesting.org>
Subject: Re: [ldv-project] Queries in running the LDV.
EXTERNAL EMAIL
Seems that the problem is caused by incorrect paths resolution to your driver. There are two possible solutions:
1) Two paths are needed in meta.json file to source files if your driver is not placed inside the Linux kernel dir. To add paths modify the meta.json file in the build base (the directory named "clade") and at the end add
"working source trees": ["path1", "path2"].
Where provide an absolute path to the directory with source files of your driver as path1 and an absolute path to the Linux kernel dir as path2.
After that reinstall build dir by executing "update" installation command.
2) It is also possible that you do not have empty.c file compiled (we assume that it is always exist in the Linux and compiled) and in this case you can go to the web interface and edit the specifications/base.json file. Here replace "scripts/mod/empty.c" by any other relative path to any compiled .c file. The path should be relative to corresponding path given in the meta.json in "working source trees" entry (if there is no "working source trees" then use path relative to "build_dir" entry).
вт, 2 июл. 2019 г. в 17:32, Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>:
Hi,
Following the whole stack trace:
Raise exception:
Traceback (most recent call last):
File "/scratch/puneet/klever/inst_dir/klever/core/core/components.py", line 427, in run
self.main()
File "/scratch/puneet/klever/inst_dir/klever/core/core/components.py", line 332, in callbacks_caller
ret = attr(*args, **kwargs)
File "/scratch/puneet/klever/inst_dir/klever/core/core/vtg/rsg/__init__.py", line 52, in generate_requirement
self.add_models(generated_models)
File "/scratch/puneet/klever/inst_dir/klever/core/core/components.py", line 332, in callbacks_caller
ret = attr(*args, **kwargs)
File "/scratch/puneet/klever/inst_dir/klever/core/core/vtg/rsg/__init__.py", line 196, in add_models
if not empty_cc:
UnboundLocalError: local variable 'empty_cc' referenced before assignment
From: Ilja Zakharov <ilja.zakharov at ispras.ru<mailto:ilja.zakharov at ispras.ru>>
Sent: Tuesday, July 2, 2019 6:57 PM
To: Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>
Cc: Alexey Khoroshilov <khoroshilov at ispras.ru<mailto:khoroshilov at ispras.ru>>; ldv-project at linuxtesting.org<mailto:ldv-project at linuxtesting.org>
Subject: Re: [ldv-project] Queries in running the LDV.
EXTERNAL EMAIL
The file looks nice. Can you send the whole stack trace next time?
Did you run "clade-all" command after the "clade-intercept" one on the step 4?
вт, 2 июл. 2019 г. в 16:16, Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>:
Following is my job.json:
{
"Class": "Verification of Linux kernel modules",
"build base": "clade",
"program": "Linux",
"version": "3.14",
"fragmentation set": "Modules",
"add targets": [
"**"
],
"specifications set": "Linux 3.14 (base)",
"requirements": [
"linux:arch:io",
"linux:drivers:clk1",
"linux:kernel:locking:mutex",
"linux:memory safety"
]
}
Do I need to make any changes?
From: Puneet Gupta
Sent: Tuesday, July 2, 2019 6:43 PM
To: Ilja Zakharov <ilja.zakharov at ispras.ru<mailto:ilja.zakharov at ispras.ru>>
Cc: Alexey Khoroshilov <khoroshilov at ispras.ru<mailto:khoroshilov at ispras.ru>>; ldv-project at linuxtesting.org<mailto:ldv-project at linuxtesting.org>
Subject: RE: [ldv-project] Queries in running the LDV.
Hi,
Thanks for the mail.
Yeah , I got the results for “linux 3.14”, but when I edited the job.json file and replaced linux 3.14 with “clade”.
I am getting the following error:
local variable 'empty_cc' referenced before assignment
Thanks,
Puneet
From: Ilja Zakharov <ilja.zakharov at ispras.ru<mailto:ilja.zakharov at ispras.ru>>
Sent: Tuesday, July 2, 2019 6:38 PM
To: Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>
Cc: Alexey Khoroshilov <khoroshilov at ispras.ru<mailto:khoroshilov at ispras.ru>>; ldv-project at linuxtesting.org<mailto:ldv-project at linuxtesting.org>
Subject: Re: [ldv-project] Queries in running the LDV.
EXTERNAL EMAIL
Hi Puneet,
Did you successfully got results for the example "Linux 3.14" as I asked you at the second step? This is an example to learn how to run Klever and it demonstrates how verification reports should look like.
In more details:
1) Goto localhost:8998
2) Open "Linux 3.14" job in the list.
3) Click Decision->Start with default values
4) Status of the job will change to "Is solving" and reports will appear if everything is installed correctly in several minutes. You will get several reports in the right part of the web-page.
Before you try to run your own driver it is better to get results for this example to ensure that all works well on your installation and then proceed to your driver. If something goes wrong please send me screenshots of your error messages.
BTW there are other steps between “clade-intercept” and step5, I hope you also have done them.
Best,
Ilja
вт, 2 июл. 2019 г. в 15:44, Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>:
Hi Ilja,
At step 3 , I was able to open the localhost:8998 , and login as manager:manager. Then I entered to my kernel module directory and ran “clade-intercept make” and executed the step5.
In web interface also , I have edited the job.json file as you mentioned. But still I am not able to see the static analysis report in the webpage.
Can you please tell me how to view the report. If you can send an example , that would be helpful too.
Thanks,
Puneet
From: Ilja Zakharov <ilja.zakharov at ispras.ru<mailto:ilja.zakharov at ispras.ru>>
Sent: Friday, June 28, 2019 2:02 PM
To: Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>
Cc: Alexey Khoroshilov <khoroshilov at ispras.ru<mailto:khoroshilov at ispras.ru>>; ldv-project at linuxtesting.org<mailto:ldv-project at linuxtesting.org>
Subject: Re: [ldv-project] Queries in running the LDV.
EXTERNAL EMAIL
Hi Puneet,
Thanks for your interest in the Klever verification framework.
To verify your separate driver you should do the following steps:
1) First, be sure that you have some kernel version sources files to compile your driver with. It is better to choose versions of Linux in between 3.14 and 4.6.7. It is possible to check a driver with a newer version of the Linux kernel but it will require additional steps. Both the driver and Kernel should be configured and compiled for x86_64 architecture preferably.
2) Install the Klever framework according to this manual: https://klever.readthedocs.io/en/latest/deploy.html
(!) Klever will need a lot of computational resources to run, so avoid installing it on a laptop. If your driver is an open source we can even give you access to a virtual machine with Klever to try it.
3) Before trying to verify your own driver just ensure that all is installed correctly. To do that open localhost:8998, log in as manager:manager and try to solve a preexisting verification job there. If all goes smoothly, go to the next step.
4) Prepare a build base for your driver. To do that just configure the kernel and your driver and build it with a prefix command:
clade-intercept <your compilation command>"
(An example: clade-intercept make -j8 M=drivers/net/usb)
Prepare the environment:
export PATH=$KLEVER_DEPLOY_DIR/klever-addons/CIF/bin/:$PATH
Where $KLEVER_DEPLOY_DIR is an absolute path to Klever installation directory. Run "cif --help" to be sure that path is correct.
Then run the second command to create the build base:
clade-all -p linux_kernel cmds.txt
After the command, you will have a directory named "clade". I refer to it as <clade dir> in the next steps below.
5. After Klever installation, you can build the kernel and create a build base.
a) Modify the configuration file that you have used for Klever installation and add there the following entry: "Klever Build Bases": ["<clade dir>"]
b) Run again installation command exactly as at installation but replace keyword "install" with "update"
6) That is almost all. Go to the web interface. Copy your test job, click "modify", choose job.json file and in the opened editor window set proper setting and identifiers of rules to check. You will need there to set fields "build base" as "clade", "requirements" as ["linux:memory safety"] (or add additional rules to check).
If you need an example write me back.
7) To uninstall Klever use installation command with command "uninstall" instead of "install".
Please, do not hesitate to write me back and ask questions.
Best,
Ilia Zakharov
чт, 27 июн. 2019 г. в 20:57, Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>:
Hi Alexey,
Thanks for your mail.
I have downloaded the klever package and managed to install it also by following the instructions given in the “local deployment” section.
https://buildmedia.readthedocs.org/media/pdf/klever/latest/klever.pdf
Can you guide me how to run the klever tool on my kernel module? Is there any doc which I can follow in which step by step the instructions are given?
Currently I have my kernel module as a tar.bz2 and I want to do static analysis on it.
I tried running ,
* cd bridge
* python3 manage.py test
I got the following error:
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/group/cdc_hd/members/puneetg/kernel_static_analysis/klever/klever-inst/klever/bridge/bridge/__init__.py", line 21, in <module>
from bridge.settings import DATABASES
ModuleNotFoundError: No module named 'bridge.settings'
I can also see “linux-3.14.79.tar.xz” tar file in “build bases” directory. Shall I need to place my external module tar file in that directory and run some tests?
Thanks,
Puneet
From: Alexey Khoroshilov <khoroshilov at ispras.ru<mailto:khoroshilov at ispras.ru>>
Sent: Friday, June 21, 2019 9:16 PM
To: Puneet Gupta <PUNEETG at xilinx.com<mailto:PUNEETG at xilinx.com>>; ldv-project at linuxtesting.org<mailto:ldv-project at linuxtesting.org>
Subject: Re: Queries in running the LDV.
EXTERNAL EMAIL
Dear Puneet,
It seems you use LDV Tools, but it was superseded by our newer KLEVER Framework [1,2].
So, we suggest to use the newer tool.
As for memory leak rule, we had a prototype of such rule in LDV Tools (104 [3]), but it was not maintained and it is not available in KLEVER. So, we suggest to check code against some memory safety rules ((KLEVER rule id: "linux:memory safety", please note that string operations are not supported yet) or some Linux specific rules like mutex lock [4] or clock_enable/clock_disable [5].
[1] http://linuxtesting.org/04-07-2018
[2] https://forge.ispras.ru/projects/klever
[3] https://forge.ispras.ru/issues/2735
[4] https://forge.ispras.ru/issues/1940 (KLEVER rule ids: "linux:kernel:locking:mutex")
[5] https://forge.ispras.ru/issues/8785 (KLEVER rule ids: "linux:drivers:clk1", "linux:drivers:clk2")
Best regards,
Alexey Khoroshilov
Linux Verification Center, ISPRAS
On 21.06.2019 8:30, Puneet Gupta wrote:
Hi,
I am using LDV for running static analysis for my external driver.
I have few questions :
* The driver has many files to be compiled, but it seems the LDV is only reporting out the errors for the file which is mentioned in ‘obj-m’. How can I get result for all the files?
* What is meant by the error: “attempting to take field offset lock of expression priv at stop whose value is Top”. I am getting only these errors for my file.
* To test the LDV , I did kmalloc and didn’t freed the memory returned. But the output on the webpage is not showing an error for that memory leak. Have I need to pass some separate rule for detecting the memory leaks?
I am using 32_7a rule for running the driver, as given in the example in the LDV standard doc.
Thanks,
Puneet
This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.
_______________________________________________
ldv-project mailing list
ldv-project at linuxtesting.org<mailto:ldv-project at linuxtesting.org>
http://linuxtesting.org/cgi-bin/mailman/listinfo/ldv-project
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://linuxtesting.org/pipermail/ldv-project/attachments/20190705/c77089d5/attachment.html>
More information about the ldv-project
mailing list