Issue921

Title [inotify] hg fetch can delete the wrong file when merging
Priority bug Status chatting
Superseder Nosy List bos, djc, jglick, kirr, kupfer, mathieu.clabaut, mpm, pmezard, tonfa
Assigned To bos Topics inotify, merge

Created on 2008-01-09.02:15:38 by jglick, last changed 2008-10-15.01:35:04 by jglick.

Files
File name Uploaded Type Edit Remove
test-fetch2 pmezard, 2008-01-12.12:40:18 application/octet-stream
test-fetch2.out pmezard, 2008-01-12.12:40:44 application/octet-stream
Messages
msg7447 (view) Author: jglick Date: 2008-10-15.01:35:03
Still broken in 6dab29f6df37 after mpm's recent changes. No "abort: outstanding
uncommitted changes" now, but final output is still "a c" and stat in
hgtest-fetch-a still shows "M b; R a". As before, replacing fetch with
pull;merge;commit produces the correct result.


Note that Hg now also prints

failed to contact inotify server: No such file or directory

after each clone, which is annoying. Improving inotify/__init__.py a bit to read:

                    ui.warn(_('failed to contact inotify server from %s: %s\n')
                             % (self._rootdir, err[-1]))

produces

failed to contact inotify server from /tmp/hgtest-fetch-a/: No such file or
directory
...
failed to contact inotify server from /tmp/hgtest-fetch-b/: No such file or
directory

This seems a gratuitous warning message - the clone has just been created, so of
course there is no server active in it yet.
msg7051 (view) Author: tonfa Date: 2008-09-10.11:18:39
djc: yes

Anyway I need to understand better what the problems are.
msg7050 (view) Author: djc Date: 2008-09-10.11:10:44
Is the print statement here for debugging?
msg7044 (view) Author: tonfa Date: 2008-09-09.22:59:39
ok, it seems inotify doesn't handle negative size and time (they have a special
meaning in the dirstate).

diff --git a/hgext/inotify/server.py b/hgext/inotify/server.py
--- a/hgext/inotify/server.py
+++ b/hgext/inotify/server.py
@@ -248,14 +248,15 @@
         try:
             type_, mode, size, time = self.repo.dirstate._map[fn][:4]
         except KeyError:
-            type_ = '?'
+            type_ = st and '?' or None
+        print type_, mode, size, time, fn, st
         if type_ == 'n':
             if not st:
                 return '!'
             st_mode, st_size, st_mtime = st
-            if size and (size != st_size or (mode ^ st_mode) & 0100):
+            if size >= 0 and (size != st_size or (mode ^ st_mode) & 0100):
                 return 'm'
-            if time != int(st_mtime):
+            if time != int(st_mtime) or size < 0:
                 return 'l'
             return 'n'
         if type_ in 'ma' and not st:
msg7039 (view) Author: jglick Date: 2008-09-09.21:14:50
Forget what I said. Hg 1.0+ apparently does not activate the inotify extension
on demand, you need to run inserve. Given that, I make up a new test script:


#!/bin/bash
killall hg
set -e -x
hg version
cd /tmp
rm -rf hgtest-fetch*
hg init hgtest-fetch
hg -R hgtest-fetch inserve &
hg clone hgtest-fetch hgtest-fetch-a
hg -R hgtest-fetch-a inserve &
hg clone hgtest-fetch hgtest-fetch-b
hg -R hgtest-fetch-b inserve &
cd hgtest-fetch-a
touch a
hg add a
hg ci -m.
hg push
cd ../hgtest-fetch-b
hg pull -u
ls
touch b
hg add b
hg rem a
hg ci -m.
ls
hg push
cd ../hgtest-fetch-a
touch c
hg add c
hg ci -m.
hg fetch --switch-parent
#hg pull
#hg merge
#hg ci -m.
ls
hg push
cd ../hgtest-fetch-b
ls
hg pull -u
ls


which gives me a different problem:


hg: no process killed
+ hg version
Mercurial Distributed SCM (version 1.0.2)

Copyright (C) 2005-2008 Matt Mackall <mpm@selenic.com> and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ cd /tmp
+ rm -rf hgtest-fetch hgtest-fetch-a hgtest-fetch-b
+ hg init hgtest-fetch
+ hg -R hgtest-fetch inserve
+ hg clone hgtest-fetch hgtest-fetch-a
updating working directory
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ hg -R hgtest-fetch-a inserve
+ hg clone hgtest-fetch hgtest-fetch-b
updating working directory
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ hg -R hgtest-fetch-b inserve
+ cd hgtest-fetch-a
+ touch a
+ hg add a
+ hg ci -m.
+ hg push
pushing to /tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
+ cd ../hgtest-fetch-b
+ hg pull -u
pulling from /tmp/hgtest-fetch
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ ls
a
+ touch b
+ hg add b
+ hg rem a
not removing a: file is modified (use -f to force removal)
+ hg ci -m.
+ ls
a  b
+ hg push
pushing to /tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
+ cd ../hgtest-fetch-a
+ touch c
+ hg add c
+ hg ci -m.
+ hg fetch --switch-parent
abort: outstanding uncommitted changes


For some reason, 'hg rem a' claims 'a' is modified when it clearly is not. Look at:

tmp$ hg -R hgtest-fetch-b st
M a
M b
tmp$ hg -R hgtest-fetch-b di
tmp$ 

I have seen this also in my own repo when trying to use inotify in Hg 1.0.2;
incorrectly says things are modified even right after I 'hg revert' them. Until
this more obvious bug is fixed, I have to assume the bug originally reported
here could still be broken. (Using 'hg rem -f a' in the test case does not help,
because it just dies later on in 'hg fetch' claiming outstanding changes.)
msg7036 (view) Author: jglick Date: 2008-09-09.20:55:29
I cannot reproduce this bug in Hg 1.0.2. (Note that the behavior of 'hg fetch'
changed in 1.0, so I tried also with --switch-parent to better mimic the 0.9.5
merge order.)

I am unfortunately unable to confirm that the bug is still reproducible in
0.9.5, because I cannot use the tip of
http://hg.kublai.com/mercurial/extensions/inotify with Hg 0.9.5 built from source:

....
  File "/space/src/hg/build/lib.linux-i686-2.5/mercurial/commands.py", line
2571, in status
    list_clean=all or opts['clean'])]
  File "/space/src/hg/build/lib.linux-i686-2.5/mercurial/localrepo.py", line
909, in status
    list_ignored, list_clean)
  File "/space/src/hg-inotify/__init__.py", line 83, in status
    list_unknown)
TypeError: status() takes exactly 5 arguments (6 given)
msg6973 (view) Author: tonfa Date: 2008-09-06.19:26:04
@jglick: can you try a more recent version ? can you still reproduce it ?
msg6972 (view) Author: tonfa Date: 2008-09-06.19:23:32
I can't reproduce here (with inotify):

+ hg pull -u
pulling from /tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 1 changes to 1 files
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ ls
b  c
msg5388 (view) Author: mpm Date: 2008-02-26.05:59:28
Degrading to bug, as disabling the extension will fix it.
msg4879 (view) Author: jglick Date: 2008-01-12.17:42:43
Ha, got it: bug only occurs when the inotify extension is enabled.
msg4877 (view) Author: pmezard Date: 2008-01-12.12:40:18
I am attaching a version of your initial script usable as a mercurial test. Can
you reproduce it when run as a test ?
msg4876 (view) Author: pmezard Date: 2008-01-12.12:38:14
I cannot reproduce it with your script and 0.9.5 on macosx 10.4, the last pull
gives me:
"""
+ cd ../hgtest-fetch-b
+ ls
b
+ hg pull -u
pulling from /private/tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 1 changes to 1 files
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ ls
b  c
"""
msg4848 (view) Author: jglick Date: 2008-01-09.02:15:36
Run:

---%<---
#!/bin/bash
set -e -x
hg version
cd /tmp
rm -rf hgtest-fetch*
hg init hgtest-fetch
hg clone hgtest-fetch hgtest-fetch-a
hg clone hgtest-fetch hgtest-fetch-b
cd hgtest-fetch-a
touch a
hg add a
hg ci -m.
hg push
cd ../hgtest-fetch-b
hg pull -u
ls
touch b
hg add b
hg rem a
hg ci -m.
ls
hg push
cd ../hgtest-fetch-a
touch c
hg add c
hg ci -m.
hg fetch
#hg pull
#hg merge
#hg ci -m.
ls
hg push
cd ../hgtest-fetch-b
ls
hg pull -u
ls
---%<---

I get:

---%<---
+ hg version
Mercurial Distributed SCM (version 0.9.5)

Copyright (C) 2005-2007 Matt Mackall <mpm@selenic.com> and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ cd /tmp
+ rm -rf hgtest-fetch hgtest-fetch-a hgtest-fetch-b
+ hg init hgtest-fetch
+ hg clone hgtest-fetch hgtest-fetch-a
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ hg clone hgtest-fetch hgtest-fetch-b
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ cd hgtest-fetch-a
+ touch a
+ hg add a
+ hg ci -m.
+ hg push
pushing to /tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
+ cd ../hgtest-fetch-b
+ hg pull -u
pulling from /tmp/hgtest-fetch
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ ls
a
+ touch b
+ hg add b
+ hg rem a
+ hg ci -m.
+ ls
b
+ hg push
pushing to /tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
+ cd ../hgtest-fetch-a
+ touch c
+ hg add c
+ hg ci -m.
+ hg fetch
pulling from /tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
merging with new head 2:25015baca70d
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
new changeset 3:2e03af81f688 merges remote changes with local
+ ls
b  c
+ hg push
pushing to /tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 1 changes to 1 files
+ cd ../hgtest-fetch-b
+ ls
b
+ hg pull -u
pulling from /tmp/hgtest-fetch
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 1 changes to 1 files
2 files updated, 0 files merged, 1 files removed, 0 files unresolved
+ ls
a  c
---%<---

For some reason the merge done by the Fetch extension affects the wrong files.
In fact the correct merge is left in the working dir but not committed:

---%<---
$ hg -R hgtest-fetch-a stat
M b
R a
---%<---

If you uncomment the three lines to use regular pull+merge+ci instead of fetch,
it works as expected; the last line shows 'b c'.
History
Date User Action Args
2008-10-15 01:35:04jglicksetnosy: mpm, bos, tonfa, kupfer, pmezard, mathieu.clabaut, jglick, kirr, djc
messages: + msg7447
2008-09-10 12:15:09tonfasettopic: + inotify
nosy: mpm, bos, tonfa, kupfer, pmezard, mathieu.clabaut, jglick, kirr, djc
2008-09-10 11:18:39tonfasetnosy: mpm, bos, tonfa, kupfer, pmezard, mathieu.clabaut, jglick, kirr, djc
messages: + msg7051
2008-09-10 11:10:45djcsetnosy: + djc
messages: + msg7050
2008-09-09 22:59:41tonfasetnosy: mpm, bos, tonfa, kupfer, pmezard, mathieu.clabaut, jglick, kirr
messages: + msg7044
2008-09-09 21:14:52jglicksetstatus: resolved -> chatting
nosy: mpm, bos, tonfa, kupfer, pmezard, mathieu.clabaut, jglick, kirr
messages: + msg7039
2008-09-09 20:55:30jglicksetstatus: chatting -> resolved
nosy: mpm, bos, tonfa, kupfer, pmezard, mathieu.clabaut, jglick, kirr
messages: + msg7036
2008-09-06 19:26:04tonfasetnosy: mpm, bos, tonfa, kupfer, pmezard, mathieu.clabaut, jglick, kirr
messages: + msg6973
2008-09-06 19:23:32tonfasetnosy: + tonfa
messages: + msg6972
2008-06-29 11:55:29kirrsetnosy: + kirr
2008-02-26 05:59:29mpmsetpriority: urgent -> bug
nosy: + mpm, bos
messages: + msg5388
assignedto: bos
2008-02-05 00:25:41kupfersetnosy: + kupfer
2008-01-12 17:42:45jglicksetnosy: pmezard, mathieu.clabaut, jglick
messages: + msg4879
title: hg fetch can delete the wrong file when merging -> [inotify] hg fetch can delete the wrong file when merging
2008-01-12 12:40:44pmezardsetfiles: + test-fetch2.out
nosy: pmezard, mathieu.clabaut, jglick
2008-01-12 12:40:18pmezardsetfiles: + test-fetch2
nosy: pmezard, mathieu.clabaut, jglick
messages: + msg4877
2008-01-12 12:38:14pmezardsetstatus: unread -> chatting
nosy: + pmezard
messages: + msg4876
2008-01-09 11:10:28mathieu.clabautsetnosy: + mathieu.clabaut
2008-01-09 02:15:38jglickcreate