MAXQDA

Large table files

26.04.2021, 10:55

Hello,

I am working with a Twitter corpus of very large excel sheets with about 160 000 tweets in total + their metadata and it has been impossible to load all three in one project. It also seems that I can't put them in the external folder.
Is there a way to work with several excel sheets in one project file or am I bound to work on separate project files ?

Thank you.

Version: MAXQDA 2020
System: Windows 10
Alice
 
Posts: 2
Joined: 23.04.2021, 15:07

Re: Large table files

27.04.2021, 08:27

The Twitter data import module of MAXQDA imports "table documents", which is different from importing Excel documents as "survey data". Tables with 1000 Excel rows are treated as one document. This seems to be easier to handle for the program than to treat each Twitter post as a separate document, as normal Excel import does. The number of documents is massively reduced.

The Twitter module has some special options for basic analysis of Twitter meta data when data is organized this way. But the possibilities for analyzing content qualitatively is more limited than in the "one document per post approach". Table columns in the "1000 posts per document" approach are less powerful for filtering and analysis than document variables are in the "survey structure" approach.
Asterix
 
Posts: 4
Joined: 10.03.2020, 22:46

Re: Large table files

27.04.2021, 17:49

Dear Asterix,
It's difficult to say for sure without doing some testing, but "very large excel sheets with about 160 000 tweets" does indeed sound like a .. performance challenge, so I would not be surprised if MAXQDA has to struggle with it.

In regard to your second post: I can't go into details as to why the "one-document-per-1k-tweets"-approach has been selected for the Twitter import/analysis features, but I agree with your assessment that using the "import excel as survey"-option is preferable in this case for the reasons you have mentioned (basically more analysis options).

Usually, using one document per case is also much better performance-wise than using one Excel file, so I'd actually use this import option as well in your case (working with Excel sheets). There are actually only very few cases where I think importing an Excel file as such is the best choice (especially for larger data sets).

Best regards on behalf of the MAXQDA support team,

Andreas
MAXQDA Support Team
Andreas V.
 
Posts: 275
Joined: 13.04.2017, 16:23

Re: Large table files

03.05.2021, 14:23

Dear Asterix & Andreas,

thank you for your answers. I am not sure I quite understand what you are suggesting.
For now I have three different excel files that constitute my Twitter corpus. Together they make about 160 000 lines which I want to analyze qualitatively. I tried to import them as such, not a survey files, but this is too heavy on the software and impossible to work with. I think importing them as survey files does not change much, I'm not sure if that was what you were suggesting.
The other suggestion I think I understood and what seems to be my only option right now is to simply work with one excel file on one MAXQDA project instead of my full corpus in the same MAXQDA project.
Alice
 
Posts: 2
Joined: 23.04.2021, 15:07

Re: Large table files

04.05.2021, 08:46

Hi Alice,

Thank you for your clarification.

Indeed, working with a table document that contains 160,000 rows can affect the performance of the software. Let me quote from our technical limitations chapter of the MAXQDA online manual: "For table documents, the time it takes to open the document depends on the number of rows/columns and the size of the cell contents. For 1,000 rows of 20 columns, it may take a few seconds for the document to open. The time to open is also affected by how many codes there are in the table. The coding and editing of cells in an open table document with 1,000 rows and 20 columns can then be completed without any delays." Therefore, the performance issues you are currently experiencing are regrettably something to be expected. I suspect that saving such a large table document as an external document wouldn't do much good either, since MAXQDA would need time to process and open the equally large number of rows in any case.

Should you need further assistance on this matter, please contact us via sending us a ticket: https://www.maxqda.com/function-support

With best regards

MAXQDA Support Team
Aikokul
 
Posts: 87
Joined: 19.07.2019, 09:48

Return to Technical Questions

Who is online

Users browsing this forum: Bing [Bot] and 5 guests

We use cookies to improve your experience on our website. By clicking OK or by continuing to browse the website, we’ll assume that you are happy with their use. Click here to review our Cookie Policy. OK