Thanks to visit codestin.com
Credit goes to github.com

Skip to content

dzsave from cli can't store multiple dzi files in the same directory #3007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
OyvindLGjesdal opened this issue Aug 23, 2022 · 16 comments
Closed

dzsave from cli can't store multiple dzi files in the same directory #3007

OyvindLGjesdal opened this issue Aug 23, 2022 · 16 comments
Labels
bug

Comments

@OyvindLGjesdal
Copy link

Questions, enhancements, tips, etc.

Please use libvips discussions for other topics.

https://github.com/libvips/libvips/discussions

Bug report

Describe the bug

When upgrading from 8.12 to 8.13 I noticed our process which generates multiple dzi to the same folder (dzi) had failed creating dzi files.

To Reproduce
When running `

vips dzsave file1 dzi/out1
vips dzsave file2 dzi/out2

or

vips dzsave file1 out1
vips dzsave file2 out2

it returns

glib: .: File exists

or

glib: dzi: File exists

and stops

Expected behavior
I would expect dzsave to still be able to write out .dzi files and subfolders to a folder that already exists, and keep the behaviour from 8.12.

Actual behavior
See error message above.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment

  • OS : Centos 7, vips installed from remi - vips.x86_64 0:8.13.0-1.el7.remi and vips-tools.8.13.0-1.el7.remi

Additional context
Normal folder structure for our output, which works for the previous version:
folder

I looked for a workaround looking at the options in the cli, but couldn't find a way to make the cli behave/create the folder structure that we are currently using.

Thanks for the awesome work on libvips!
@jcupitt
Copy link
Member

jcupitt commented Aug 23, 2022

Hi @OyvindLGjesdal,

Oooof that's bad, I'll have a look. I think this is a side-effect of the new dzsave-to-a-pipe feature.

@jcupitt
Copy link
Member

jcupitt commented Aug 23, 2022

It's working for me:

john@banana ~/pics/x $ vips --version
vips-8.13.1
john@banana ~/pics/x $ rm -rf * 
john@banana ~/pics/x $ ls
john@banana ~/pics/x $ vips dzsave ../k2.jpg x
john@banana ~/pics/x $ vips dzsave ../k2.jpg y
john@banana ~/pics/x $ ls
x.dzi  x_files  y.dzi  y_files
john@banana ~/pics/x $

Could it be the libgsf version you're using? I'm on 1.14.47.

@OyvindLGjesdal
Copy link
Author

There is a big difference with the version number of libgsf: libgsf.x86_64 1.14.26-7.el7 @base

Replicating your example:

[centos@marcus x]$ rm -rf *
[centos@marcus x]$ vips dzsave ../k2.tif x

(vips:19726): VIPS-WARNING **: 12:34:35.282: Incompatible type for "RichTIFFIPTC"; tag ignored

(vips:19726): VIPS-WARNING **: 12:34:35.283: Incompatible type for "RichTIFFIPTC"; tag ignored

(vips:19726): VIPS-WARNING **: 12:34:35.283: Incompatible type for "RichTIFFIPTC"; tag ignored

(vips:19726): VIPS-WARNING **: 12:34:35.283: Incompatible type for "RichTIFFIPTC"; tag ignored

(vips:19726): VIPS-WARNING **: 12:34:35.283: Incompatible type for "RichTIFFIPTC"; tag ignored
glib: .: File exists

[centos@marcus x]$ vips dzsave ../k2.tif y

(vips:19727): VIPS-WARNING **: 12:34:58.144: Incompatible type for "RichTIFFIPTC"; tag ignored

(vips:19727): VIPS-WARNING **: 12:34:58.145: Incompatible type for "RichTIFFIPTC"; tag ignored

(vips:19727): VIPS-WARNING **: 12:34:58.145: Incompatible type for "RichTIFFIPTC"; tag ignored

(vips:19727): VIPS-WARNING **: 12:34:58.145: Incompatible type for "RichTIFFIPTC"; tag ignored

(vips:19727): VIPS-WARNING **: 12:34:58.145: Incompatible type for "RichTIFFIPTC"; tag ignored
glib: .: File exists

[centos@marcus x]$ ls
[centos@marcus x]$ vips --version
vips-8.13.0-Thu Jul 21 13:59:28 UTC 2022
[centos@marcus x]$

@jcupitt
Copy link
Member

jcupitt commented Aug 23, 2022

I think you'll need to update your libgsf to fix this.

@OyvindLGjesdal
Copy link
Author

Thank you for the swift response and check @jcupitt. Understandable that things which are broken in old dependencies can't and won't be fixed. I think maybe we stay at vips-8.12.2-Tue for now since I've not had much experience or luck with replacing core libraries.

@jcupitt
Copy link
Member

jcupitt commented Aug 23, 2022 •
edited
Loading

Don't replace the library in /usr/lib, just build your own libgsf alongside your own libvips. I made a dockerfile for you:

https://github.com/jcupitt/docker-builds/blob/master/libvips-centos7.9/Dockerfile

I see:

Removing intermediate container b1f6fabb4b3d
 ---> 60b0351d89da
Step 14/14 : RUN ldd /usr/local/bin/vips
 ---> Running in 6a17c70fabcc
	linux-vdso.so.1 =>  (0x00007fff8d1f8000)
	libvips.so.42 => /usr/local/lib/libvips.so.42 (0x00007f2e8dbd2000)
	libexpat.so.1 => /lib64/libexpat.so.1 (0x00007f2e8d9a8000)
	libgsf-1.so.114 => /usr/local/lib/libgsf-1.so.114 (0x00007f2e8d762000)
	libxml2.so.2 => /lib64/libxml2.so.2 (0x00007f2e8d3f8000)
	libfftw3.so.3 => /lib64/libfftw3.so.3 (0x00007f2e8d073000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f2e8ce5d000)
	libpng15.so.15 => /lib64/libpng15.so.15 (0x00007f2e8cc32000)
	libjpeg.so.62 => /lib64/libjpeg.so.62 (0x00007f2e8c9dd000)
	libexif.so.12 => /lib64/libexif.so.12 (0x00007f2e8c797000)
	libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f2e8c593000)
	libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007f2e8c1f3000)
	libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007f2e8bfa2000)
	libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f2e8bc8c000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f2e8b98a000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2e8b76e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f2e8b3a0000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2e8b098000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2e8ae82000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f2e8ac7e000)
	liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f2e8aa58000)
	libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f2e8a7f6000)
	libffi.so.6 => /lib64/libffi.so.6 (0x00007f2e8a5ee000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f2e8a3c7000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f2e8a1ad000)
	libmount.so.1 => /lib64/libmount.so.1 (0x00007f2e89f6a000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f2e8e21a000)
	libblkid.so.1 => /lib64/libblkid.so.1 (0x00007f2e89d2a000)
	libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f2e89b25000)
Removing intermediate container 6a17c70fabcc
 ---> 36669358f143
Successfully built 36669358f143
Successfully tagged libvips-centos7.9:latest

You can see it's using the custom libgsf 1.14.45 in /usr/local/lib, not the system one.

@remicollet
Copy link
Contributor

why is this closed ?

@jcupitt isn't possible to fix this ?

@jcupitt
Copy link
Member

jcupitt commented Aug 30, 2022

Hi @remicollet,

libvips 8.12 had a complicated workaround for old and broken libgsf. The new save-dz-to-a-pipe feature makes this difficult to support now.

It should work with libgsf-1.14.27, from 2013. Are there many users still on this old version?

@remicollet
Copy link
Contributor

See https://rpms.remirepo.net/rpmphp/zoom.php?rpm=libgsf

So RHEL/CentOS 7 (and other clones) are still using 1.14.26

EL-7 is 8 years old and have enter "Maintaince support phase 2" (only critical issues)

I'm used to encourage users to use a modern distro for modern features (EL-8 or even EL-9), but it seems lot of users are still using old EL-7 (for bad / strange reasons)

Example for download statistics for vips 8.13 (>6k download):

  • EL-7: 78% :(
  • EL-8: 19/%
  • EL-9: 3%

@jcupitt
Copy link
Member

jcupitt commented Aug 30, 2022

Eeek!

OK, I'll see if there's some way we can hack support back in.

@jcupitt jcupitt reopened this Aug 30, 2022
@kleisauke
Copy link
Member

It looks like this was fixed in libgsf with commit GNOME/libgsf@dfd8750 (version 1.14.29), upstream bug report:
https://bugzilla.gnome.org/show_bug.cgi?id=706219

@jcupitt
Copy link
Member

jcupitt commented Aug 30, 2022

Ah not 1.14.27 then, thanks Kleis.

Yes, the workaround was to always write to a subdirectory, then rename all the files in a second pass outside libgsf. This is obviously tricky when writing to a pipe, since you can't do a second pass.

@kleisauke
Copy link
Member

Another option is to implement GsfOutfileStdio by ourselves, see e.g. commit kleisauke@b1f5a29. I could open a PR for that (targeting the 8.13 branch), but perhaps it should only be done conditionally (libgsf <= 1.14.29)?

@jcupitt
Copy link
Member

jcupitt commented Aug 30, 2022

That's a good idea!

Let's always use our own stdio writer and reduce the number of cases we need to test.

@kleisauke kleisauke mentioned this issue Aug 30, 2022
Merged
@kleisauke
Copy link
Member

I just opened PR #3017 for this.

@jcupitt
Copy link
Member

jcupitt commented Aug 30, 2022

... and merged. That's a very neat solution Kleis.

@jcupitt jcupitt closed this as completed Aug 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@remicollet @jcupitt @OyvindLGjesdal @kleisauke