Problems and Technical Issues with Rosetta@home
Message boards : Number crunching : Problems and Technical Issues with Rosetta@home
Previous · 1 . . . 145 · 146 · 147 · 148 · 149 · 150 · 151 . . . 300 · Next
| Author | Message |
|---|---|
|
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 187 Credit: 6,384,406 RAC: 5,531 
|
I have the following in my app_config file: [/var/lib/boinc/projects/boinc.bakerlab.org_rosetta]$ cat app_config.xml <app_config> <project_max_concurrent>3</project_max_concurrent> </app_config> and the boinc-cllient runs up to three of these work units at a time. I am getting plenty of work units. I have a 16-core machine but the boinc-client is instructed to run no more than 8 Boinc tasks at a time. The various app_config.xml files allow up to 4 ClimatePrediction tasks at a time, 5 WCG at a time, three Rosetta at a time, and 2 Universe tasks at a time -- though the priority of the Universe tasks is so low that they hardly ever run, and then only one at a time. Currently I have these Rosetta tasks on my machine and none of them are running. 3443919 Dec 9 13:09 fcgr3a_site1_5mn2_3h_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_7ts1jp9r.zip 3453466 Dec 9 13:09 fcgr3a_site1_5mn2_3h_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_9zi4uu3j.zip 2646638 Dec 9 13:09 her2_site4_3h_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_0ic4nn1s.zip 2544075 Dec 9 13:09 her2_site4_3h_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_6kw0cf1j.zip 2804583 Dec 9 13:09 niv_site3_6vy5_jhr_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_2fs1jz6r.zip 2585084 Dec 9 13:09 niv_site3_6vy6_jhr_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_2cm7fe4w.zip 2748992 Dec 9 13:09 niv_site3_6vy6_jhr_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_2jg7fo9z.zip 2640314 Dec 9 13:09 niv_site3_6vy6_jhr_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_7jp7ej6d.zip 2575060 Dec 9 13:09 niv_site3_6vy6_jhr_ggraft_1_SAVE_ALL_OUT_IGNORE_THE_REST_8yo1qr6j.zip This is when I have been receiving work units for Rosetta: Tue 07 Dec 2021 03:10:29 PM EST | Rosetta@home | Sending scheduler request: To fetch work. Tue 07 Dec 2021 03:10:29 PM EST | Rosetta@home | Requesting new tasks for CPU Tue 07 Dec 2021 03:10:33 PM EST | Rosetta@home | Scheduler request completed: got 9 new tasks Wed 08 Dec 2021 12:30:55 AM EST | Rosetta@home | Sending scheduler request: To fetch work. Wed 08 Dec 2021 12:30:55 AM EST | Rosetta@home | Requesting new tasks for CPU Wed 08 Dec 2021 12:30:59 AM EST | Rosetta@home | Scheduler request completed: got 8 new tasks Wed 08 Dec 2021 09:52:40 AM EST | Rosetta@home | Sending scheduler request: To fetch work. Wed 08 Dec 2021 09:52:40 AM EST | Rosetta@home | Reporting 2 completed tasks Wed 08 Dec 2021 09:52:40 AM EST | Rosetta@home | Requesting new tasks for CPU Wed 08 Dec 2021 09:52:42 AM EST | Rosetta@home | Scheduler request completed: got 8 new tasks Wed 08 Dec 2021 10:55:08 PM EST | Rosetta@home | Sending scheduler request: To fetch work. Wed 08 Dec 2021 10:55:08 PM EST | Rosetta@home | Reporting 3 completed tasks Wed 08 Dec 2021 10:55:08 PM EST | Rosetta@home | Requesting new tasks for CPU Wed 08 Dec 2021 10:55:16 PM EST | Rosetta@home | Scheduler request completed: got 9 new tasks I do not know about python tasks because I do not get them because I cannot run them. 
|
 Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Something different...I have been getting some 4.2 stuff and now I am back to Python. I think you missed out at what I am asking. I get both 4.2 and Python. However, I had to limit RAH to 2 tasks at a time because while 4.2 runs just fine and takes very little memory and processes very fast, Python is a memory hog and if it runs 3 it kills my other projects ability to run seamlessly. So I was wondering if anyone has seen a pattern from the two schedulers, does it do 4.2 for a time and then go back to Python or does it also go 1:1 or whatever with python and 4.2 at the same time? Or is it just a random mix of the two? |
|
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Before I got interested in python I let RAH take whatever it wanted. Then when I got into Python and letting it do its thing, then it was 3 tasks and taking up over half the memory and stalling other tasks. Then I took your project_concurrent and limited it to 2. But I see "name" does not affect anything. Project is project. So RAH is limited to 2 tasks no matter which application is running. I am afraid to release it, because then Python will take over. Unless that was a matter of RAH debt when 4.2 ran out. All I know is that Python x3 is a system killer unless I buy some new memory sticks. Then we are talking an absurd amount of memory to keep BOINC running the way I like it to run or dropping Python which seems crazy after complaining about not getting it for so long. Why can't you run Python? I forgot. |
|
.clair. Send message Joined: 2 Jan 07 Posts: 274 Credit: 26,399,595 RAC: 0 |
All I know is that Python is a system killer unless I buy some new memory sticks. I don't know what motherboard you have, but don't scimp on it, you may have to max out what the board can take. ................ Recently because of going to some dodgy website an downloading a new version of boinc mangler 7,16.20 that had `virtual pox` in with it. {its not had VB before} I get my computer infected with pythons in cages, and they are horrible big things that realy take over the poor thing. I decided to upgrade one my systems from 32GB to 128GB {not its max} it`s `only` a twin xeon E5 2697 48 thread 128GB of ram and they can realy stuff it. even if actual memory use is 50GB and only run 16 at a time. it'll run plenty more R4.20, [ and other work] further up this thread I had problems with another system getting daft disk space messages that I now ignore, {and I had just upped that from 16 to 32 GB} now even this one gets them, Just now, rosetta is using 77GB of disk space and it has 246GB more free available for boinc to use. ............. 09-Dec-2021 06:40:46 [Rosetta@home] Rosetta needs 1907.35MB more disk space. You currently have 0.00 MB available and it needs 1907.35 MB. 09-Dec-2021 06:40:46 [Rosetta@home] rosetta python projects needs 19073.49MB more disk space. You currently have 0.00 MB available and it needs 19073.49 MB ............ crazy stuff I don't bother about any more And I have rosetta set to a zero resource share on that system to try and limit what funky stuff it can do |
|
Sid Celery Send message Joined: 11 Feb 08 Posts: 2118 Credit: 41,165,101 RAC: 14,045
|
Looks like another batch of 4.2 work is available. I did get a few Robetta tasks that ran the full 8hrs but it looks like all Rosetta 4.20 tasks have been downloaded now. Won't be long before I'm back running WCG My resource share is set to Rosetta 29 - WCG 1 but my RAC is 6.5k to 19.6k atm... <sigh>  
|
|
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 187 Credit: 6,384,406 RAC: 5,531
|
Why can't you run Python? I forgot. I do not have VirtualBox, so I cannot run them.
|
|
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Why can't you run Python? I forgot. Do you want to run them? Then download virtual box 5 not 6. And enable the option to get them on your profile on this page: https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=5958977 (Thats your windows machine) But they are memory hogs (can only do 3 tasks) plus the VDI is 9 gigs. |
|
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Why can't you run Python? I forgot. Each Python requires 7629.39 MB of physical memory and an additional 101 to 103 MB of Virtual Memory. Which is why I put the restriction in place so other tasks from other projects could run. |
|
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 187 Credit: 6,384,406 RAC: 5,531
|
While I have python, I do not want to bother installing and running VirtualBox on my machine; it has enough to do already. [/usr/bin]$ ls -l python* lrwxrwxrwx. 1 root root 9 Aug 11 06:22 python2 -> python2.7 -rwxr-xr-x. 1 root root 8016 Aug 11 06:22 python2.7 lrwxrwxrwx. 1 root root 25 Oct 26 2020 python3 -> /etc/alternatives/python3 lrwxrwxrwx. 1 root root 31 Aug 11 06:56 python3.6 -> /usr/libexec/platform-python3.6 lrwxrwxrwx. 1 root root 32 Aug 11 06:56 python3.6m -> /usr/libexec/platform-python3.6m
|
|
Falconet Send message Joined: 9 Mar 09 Posts: 353 Credit: 1,227,479 RAC: 2,728 
|
And more work was added to the queue. From 2.2 million to 6 million tasks. |
|
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1673 Credit: 17,663,319 RAC: 22,707 
|
And more work was added to the queue.many of these are completing in 1 hour or so, not the 8 hr Target CPU time. So they won't last long at all if most of them are like these initial ones. Grant Darwin NT |
|
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Well you can exclude them if you don't want to have them. Same page allows you to opt out. |
|
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 187 Credit: 6,384,406 RAC: 5,531
|
While I have python, I do not want to bother installing and running VirtualBox on my machine; it has enough to do already. To what page do you refer? OTOH, for some reason, I do not get any, though I do get the messages Fri 10 Dec 2021 10:30:09 AM EST | Rosetta@home | Message from server: VirtualBox is not installed <---<<< Fri 10 Dec 2021 10:30:09 AM EST | Rosetta@home | Project requested delay of 31 seconds Fri 10 Dec 2021 12:26:12 PM EST | Rosetta@home | Sending scheduler request: To fetch work. Fri 10 Dec 2021 12:26:12 PM EST | Rosetta@home | Requesting new tasks for CPU Fri 10 Dec 2021 12:26:15 PM EST | Rosetta@home | Scheduler request completed: got 9 new tasks
|
|
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1673 Credit: 17,663,319 RAC: 22,707
|
To what page do you refer?The options on the page mentioned aren't there because you don't have VirtualBox installed. OTOH, for some reason, I do not get any, though I do get the messagesJust as the message says- because you don't have VirtualBox installed, you won't get any. If you want to do Python Tasks, you must install VirtualBox. If you don't want to do them, or just don't want to install VirtualBox, then don't. Grant Darwin NT |
|
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
To what page do you refer?The options on the page mentioned aren't there because you don't have VirtualBox installed. Thanks for the assist. I forgot that non Vbox users can't see the tasks. But then again, why should they get the error saying Vbox is not installed? IF they were smart, then when they detect that Vbox is not installed, just ignore sending any messages to the system. Would save this kind of confusion. |
|
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Looks like another batch of 4.2 work is available. Sid - not all of that millions is for us. I randomly sampled some of the jobs and found some with RoseTTAFold which is the neural network. There is no way on that page to isolate those from the rest without clicking on each one. Like this group: Potassium-transporting ATPase alpha chain 2 11 That is all for RoseTTaFold. And so on. |
|
Falconet Send message Joined: 9 Mar 09 Posts: 353 Credit: 1,227,479 RAC: 2,728
|
And more work was added to the queue.many of these are completing in 1 hour or so, not the 8 hr Target CPU time. So they won't last long at all if most of them are like these initial ones. Some of my tasks are running beyond the 100 decoy mark. For example, this one. |
|
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
And more work was added to the queue.many of these are completing in 1 hour or so, not the 8 hr Target CPU time. So they won't last long at all if most of them are like these initial ones. Try this one ....198 |
|
Kompakki Send message Joined: 14 Jul 14 Posts: 3 Credit: 19,224,841 RAC: 18,442
|
I have encountered some virtualbox tasks that takes over a day to run. When task takes over a day to run, I thought that it is stuck in some way and aborted the task. Other thought is that there is some problem in my environment. Does virtualbox tasks usually take over a day to run? Is there some problem in my environment? One longlasting task is as follows: Task 1453463295 Name aagb-mALA_pp-mHPR-mACPenC13T-B3PHE_1_2611271_1_0 Workunit 1297021861 Created 8 Dec 2021, 21:03:21 UTC Sent 9 Dec 2021, 13:51:23 UTC Report deadline 12 Dec 2021, 13:51:23 UTC Received 11 Dec 2021, 12:36:28 UTC Server state Over Outcome Computation error Client state Aborted by user Exit status 203 (0x000000CB) EXIT_ABORTED_VIA_GUI Computer ID 2401895 Run time 1 days 2 hours 45 min 39 sec CPU time 25 min 55 sec Validate state Invalid Credit 14.00 Device peak FLOPS 4.90 GFLOPS Application version rosetta python projects v1.03 (vbox64) x86_64-pc-linux-gnu Peak working set size 1,185.08 MB Peak swap size 2,565.54 MB Peak disk usage 7,979.68 MB Computer: IP address Show IP address Domain name huono3-desktop Local Standard Time UTC +2 hours Created 17 Nov 2015, 17:05:25 UTC Total credit 4,147,395 Average credit 262.50 Cross project credit BOINCstats.com Free-DC CPU type GenuineIntel Intel(R) Pentium(R) CPU G3258 @ 3.20GHz [Family 6 Model 60 Stepping 3] Number of processors 2 Coprocessors --- Operating System Linux Ubuntu Ubuntu 20.04.3 LTS [5.4.0-91-generic|libc 2.31 (Ubuntu GLIBC 2.31-0ubuntu9.2)] BOINC version 7.16.6 Memory 7639 MB Cache 3072 KB Swap space 3905 MB Total disk space 106.16 GB Free Disk Space 83.67 GB Measured floating point speed 4895.56 million ops/sec Measured integer speed 111568.63 million ops/sec Average upload rate 160.08 KB/sec Average download rate 2790.46 KB/sec Average turnaround time 2.77 days Application details Show Tasks 27 Number of times client has contacted server 5500 Last time contacted server 14 Dec 2021, 12:32:40 UTC Fraction of time BOINC is running 83.79% While BOINC is running, fraction of time computing is allowed 100.00% While is BOINC running, fraction of time GPU computing is allowed 100.00% Task duration correction factor 1 virtual box version: 6.1.26 |
|
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
I have encountered some virtualbox tasks that takes over a day to run. When task takes over a day to run, I thought that it is stuck in some way and aborted the task. Other thought is that there is some problem in my environment. There are a few "0 CPU" tasks. Just check the CPU usage. If it is very low, abort that task. I use BoincTasks to check that easily, but you can look at the CPU runtime in BOINC manager as well. |
Message boards :
Number crunching :
Problems and Technical Issues with Rosetta@home
©2024 University of Washington
https://www.bakerlab.org